### Navigate to Installation Folder (Windows) Source: https://github.com/haidra-org/horde-worker-regen/blob/main/README.md Use this command in PowerShell or Command Prompt to change to the desired directory for worker installation. Avoid using spaces in the installation path. ```cmd cd C:\path\to\install\folder ``` -------------------------------- ### Install WSL and Ubuntu 24.04 Source: https://github.com/haidra-org/horde-worker-regen/blob/main/README_advanced.md Installs the Windows Subsystem for Linux and the Ubuntu 24.04 distribution. This command may require running 'wsl --install' first if WSL is not already enabled. ```bash wsl --install -d Ubuntu-24.04 ``` -------------------------------- ### Enable DXG Detection and Verify ROCm Setup Source: https://github.com/haidra-org/horde-worker-regen/blob/main/README_advanced.md Commands to set environment variables for DXG detection and ROCm path, then reload the bash profile. Finally, run rocminfo to verify the setup. ```bash # 1. Add the environment variables to your bash profile echo 'export HSA_ENABLE_DXG_DETECTION=1' >> ~/.bashrc echo 'export PATH=/opt/rocm/bin:$PATH' >> ~/.bashrc # 2. Reload your profile source ~/.bashrc ``` ```bash rocminfo ``` -------------------------------- ### rocminfo Output Example Source: https://github.com/haidra-org/horde-worker-regen/blob/main/README_advanced.md Example output from the rocminfo command, indicating a successful WSL environment detection and HSA system attributes. ```text WSL environment detected. ===================== HSA System Attributes ===================== Runtime Version: 1.1 Runtime Ext Version: 1.6 System Timestamp Freq.: 1000.000000MHz Sig. Max Wait Duration: 18446744073709551615 (0xFFFFFFFFFFFFFFFF) (timestamp count) Machine Model: LARGE System Endianness: LITTLE Mwaitx: DISABLED DMAbuf Support: NO ========== HSA Agents ========== ... ``` -------------------------------- ### Start or Monitor Running Container Source: https://github.com/haidra-org/horde-worker-regen/blob/main/Dockerfiles/README.md Start a container named 'reGen' and attach to its output, or monitor a running container. CTRL+C detaches the container, leaving it running in the background. ```bash docker start -ai reGen ``` -------------------------------- ### Setup Python Virtual Environment Source: https://github.com/haidra-org/horde-worker-regen/blob/main/README_advanced.md Create and activate a Python virtual environment for the worker. This isolates dependencies and ensures consistent behavior. ```bash python -m venv regen ``` ```bash py -3.11 -m venv regen ``` ```bash regen\Scripts\activate.bat ``` ```bash regen\Scripts\Activate.ps1 ``` ```bash source regen/bin/activate ``` -------------------------------- ### Clone and Build librocdxg Library Source: https://github.com/haidra-org/horde-worker-regen/blob/main/README_advanced.md Steps to clone the librocdxg repository, set the Windows SDK path, configure the build environment using CMake, and compile/install the library. Ensure the WIN_SDK path matches your installed Windows SDK version. ```bash # 1. Clone the repository to your local WSL git clone https://github.com/ROCm/librocdxg.git cd librocdxg # 2. Set the Windows SDK path # IMPORTANT: Change "10.0.26100.0" below to match the folder version you found in Step 1! export win_sdk='/mnt/c/Program Files (x86)/Windows Kits/10/Include/10.0.26100.0/' # 3. Configure the build environment mkdir -p build cd build cmake .. -DWIN_SDK="${win_sdk}/shared" # 4. Compile and install the library make sudo make install ``` -------------------------------- ### Start a Container Detached Source: https://github.com/haidra-org/horde-worker-regen/blob/main/Dockerfiles/README.md Start a container named 'reGen' in detached mode, allowing it to run in the background. ```bash docker start reGen ``` -------------------------------- ### Install Ubuntu 24.04 Build Dependencies Source: https://github.com/haidra-org/horde-worker-regen/blob/main/README_advanced.md Installs essential build tools, CMake, Git, and SSL certificates within the Ubuntu 24.04 WSL environment. These are necessary for compiling code and managing repositories. ```bash sudo apt update && sudo apt install -y build-essential cmake git ca-certificates wget gpg ``` -------------------------------- ### Install Dependencies for RoCM Source: https://github.com/haidra-org/horde-worker-regen/blob/main/README_advanced.md Install the required Python packages for the worker, specifically for RoCM-enabled systems. This command should be run after cloning the repository and activating the virtual environment. ```bash pip install -r requirements.txt --extra-index-url https://download.pytorch.org/whl/rocm6.2 ``` -------------------------------- ### Install Dependencies for CUDA Source: https://github.com/haidra-org/horde-worker-regen/blob/main/README_advanced.md Install the required Python packages for the worker, specifically for CUDA-enabled systems. This command should be run after cloning the repository and activating the virtual environment. ```bash pip install -r requirements.txt --extra-index-url https://download.pytorch.org/whl/cu128 ``` -------------------------------- ### Build and Run with Docker Compose Source: https://github.com/haidra-org/horde-worker-regen/blob/main/Dockerfiles/README.md Build the Docker image and start the horde-worker-reGen container using Docker Compose. Replace '[cuda|rocm]' with your specific GPU type (cuda or rocm). Ensure bridgeData.yaml is configured correctly. ```bash docker compose -f Dockerfiles/compose.[cuda|rocm].yaml build --pull docker compose -f Dockerfiles/compose.[cuda|rocm].yaml up -dV ``` -------------------------------- ### Configure ROCm Repository for Ubuntu 24.04 Source: https://github.com/haidra-org/horde-worker-regen/blob/main/README_advanced.md This multi-step process adds the AMD ROCm GPG key, configures the ROCm repository for Ubuntu 24.04 (Noble), and sets an APT priority pin to ensure AMD's packages are preferred. Finally, it updates the package list and installs the ROCm stack. ```bash # 1. Download and add the AMD GPG key sudo mkdir --parents --mode=0755 /etc/apt/keyrings wget https://repo.radeon.com/rocm/rocm.gpg.key -O - | gpg --dearmor | sudo tee /etc/apt/keyrings/rocm.gpg > /dev/null # 2. Add the ROCm repository specifically for Ubuntu 24.04 (Noble) sudo tee /etc/apt/sources.list.d/rocm.list << EOF deb [arch=amd64 signed-by=/etc/apt/keyrings/rocm.gpg] https://repo.radeon.com/rocm/apt/latest noble main EOF # 3. Set the Apt priority pin to prefer AMD's repo over Ubuntu's defaults echo -e "Package: * Pin: release o=repo.radeon.com Pin-Priority: 600" | sudo tee /etc/apt/preferences.d/rocm-pin-600 # 4. Update your package list and install the ROCm stack sudo apt update sudo apt install -y rocm rocm-dev ``` -------------------------------- ### Run the Worker Source: https://github.com/haidra-org/horde-worker-regen/blob/main/README_advanced.md Start the Horde Worker Regen process. Ensure your configuration file (bridgeData.yaml) is set up with an API key and worker name. ```bash python run_worker.py ``` -------------------------------- ### Clone Horde Worker Repository (Windows/Linux) Source: https://github.com/haidra-org/horde-worker-regen/blob/main/README.md Clone the Horde worker repository using Git. This is the recommended method for installation on both Windows and Linux systems. ```git git clone https://github.com/Haidra-Org/horde-worker-reGen.git cd horde-worker-reGen ``` -------------------------------- ### Clone Horde-Worker-ReGen Repository Source: https://github.com/haidra-org/horde-worker-regen/blob/main/Dockerfiles/README.md Clone the specific directory for horde-worker-reGen using sparse checkout. This is the initial step to get the necessary Dockerfiles and configuration templates. ```bash git clone --sparse https://github.com/Haidra-Org/horde-worker-reGen.git cd horde-worker-reGen git sparse-checkout set --no-cone Dockerfiles /bridgeData_template.yaml ``` -------------------------------- ### Start Multiple GPU Workers on Linux Source: https://github.com/haidra-org/horde-worker-regen/blob/main/README.md When using multiple GPUs on Linux, start a separate worker instance for each GPU by specifying the CUDA_VISIBLE_DEVICES environment variable. Ensure sufficient RAM is available, as queue_size and max_threads impact memory usage. ```bash CUDA_VISIBLE_DEVICES=0 ./horde-bridge.sh -n "Instance 1" CUDA_VISIBLE_DEVICES=1 ./horde-bridge.sh -n "Instance 2" ``` -------------------------------- ### Download Models Source: https://github.com/haidra-org/horde-worker-regen/blob/main/README_advanced.md Execute the script to download necessary models. This is a critical first step before running the worker. ```bash python download_models.py ``` -------------------------------- ### Build Arguments for Working with Git Forks Source: https://github.com/haidra-org/horde-worker-regen/blob/main/Dockerfiles/README.md Configure Git owner and branch to easily test changes from forks or personal branches. ```bash --build-arg GIT_OWNER=your-github-username \ --build-arg GIT_BRANCH=your-feature-branch ``` -------------------------------- ### Build Arguments for CI/CD Git Integration Source: https://github.com/haidra-org/horde-worker-regen/blob/main/Dockerfiles/README.md Automate Git branch and owner configuration in CI/CD pipelines using environment variables. ```bash --build-arg GIT_BRANCH=${CI_COMMIT_BRANCH} \ --build-arg GIT_OWNER=${CI_PROJECT_NAMESPACE} ``` -------------------------------- ### Build NVIDIA (CUDA) Docker Image Source: https://github.com/haidra-org/horde-worker-regen/blob/main/Dockerfiles/README.md Builds the Docker image for NVIDIA GPUs using Dockerfile.cuda. Customize with build arguments like CUDA_VERSION and PYTHON_VERSION. ```bash docker build -f Dockerfile.cuda \ --build-arg CUDA_VERSION=12.4.1 \ --build-arg PYTHON_VERSION=3.11 \ --build-arg GIT_BRANCH=main \ --build-arg GIT_OWNER=Haidra-Org \ --build-arg USE_PIP_CACHE=true \ -t horde-worker-regen:cuda . ``` -------------------------------- ### System RAM Configuration Source: https://github.com/haidra-org/horde-worker-regen/blob/main/README.md Settings for systems with less than 32GB of System RAM. Focuses on using SD15 models and managing memory. ```yaml - load_large_models: false ``` -------------------------------- ### Build AMD (ROCm) Docker Image Source: https://github.com/haidra-org/horde-worker-regen/blob/main/Dockerfiles/README.md Builds the Docker image for AMD GPUs using Dockerfile.rocm. Customize with build arguments like ROCM_VERSION and PYTHON_VERSION. ```bash docker build -f Dockerfile.rocm \ --build-arg ROCM_VERSION=6.1.2 \ --build-arg PYTHON_VERSION=3.11 \ --build-arg GIT_BRANCH=main \ --build-arg GIT_OWNER=Haidra-Org \ --build-arg USE_PIP_CACHE=true \ -t horde-worker-regen:rocm . ``` -------------------------------- ### Generate .env file from bridgeData.yaml (Windows) Source: https://github.com/haidra-org/horde-worker-regen/blob/main/Dockerfiles/README.md Use the `convert_config_to_env.py` script with `runtime.cmd` to convert a `bridgeData.yaml` file into a `.env` file suitable for Docker. ```batch .\runtime.cmd python -s -m convert_config_to_env --file .\bridgeData.yaml ``` -------------------------------- ### Generate .env file from bridgeData.yaml (Linux) Source: https://github.com/haidra-org/horde-worker-regen/blob/main/Dockerfiles/README.md Use the `convert_config_to_env.py` script with `runtime.sh` to convert a `bridgeData.yaml` file into a `.env` file suitable for Docker. ```shell ./runtime.sh python -s -m convert_config_to_env --file ./bridgeData.yaml ``` -------------------------------- ### Generate .env file from bridgeData.yaml (venv) Source: https://github.com/haidra-org/horde-worker-regen/blob/main/Dockerfiles/README.md Execute the `convert_config_to_env.py` script directly to convert a `bridgeData.yaml` file into a `.env` file. This method is suitable for users within a virtual environment. ```python python -m convert_config_to_env --file ./bridgeData.yaml ``` -------------------------------- ### Build Argument for Pull Request Review Source: https://github.com/haidra-org/horde-worker-regen/blob/main/Dockerfiles/README.md Allows reviewers to build and test changes from a contributor's specific branch and username. ```bash --build-arg GIT_OWNER=contributor-username \ --build-arg GIT_BRANCH=feature-branch ``` -------------------------------- ### Run NVIDIA (CUDA) Container Source: https://github.com/haidra-org/horde-worker-regen/blob/main/Dockerfiles/README.md Runs the Horde Worker Regen container with CUDA support, enabling all available GPUs. ```bash docker run -it --gpus all horde-worker-regen:cuda ``` -------------------------------- ### 24GB+ VRAM Worker Configuration Source: https://github.com/haidra-org/horde-worker-regen/blob/main/README.md Recommended settings for GPUs with 24GB or more VRAM (e.g., 3090, 4090). Adjust max_threads and related settings for optimal performance. ```yaml - queue_size: 1 # <32GB RAM: 0, 32GB: 1, >32GB: 2 - safety_on_gpu: true - high_memory_mode: true - high_performance_mode: true - unload_models_from_vram_often: false - max_threads: 1 # 2 is often viable for xx90 cards - post_process_job_overlap: true - queue_size: 2 # Set to 1 if max_threads: 2 - max_power: 64 # Reduce if max_threads: 2 - max_batch: 8 # Increase if max_threads: 1, decrease if max_threads: 2 - allow_sdxl_controlnet: true ``` -------------------------------- ### Rebuild Docker Image for Worker Update Source: https://github.com/haidra-org/horde-worker-regen/blob/main/Dockerfiles/README.md Rebuild the Docker image using the appropriate Dockerfile for CUDA or AMD, and tag it accordingly. ```bash docker build -f Dockerfile.[cuda|amd] -t horde-worker-regen:[cuda|rocm] . ``` -------------------------------- ### Run AMD (ROCm) Container Source: https://github.com/haidra-org/horde-worker-regen/blob/main/Dockerfiles/README.md Runs the Horde Worker Regen container with ROCm support, mapping necessary devices and group for AMD GPU access. ```bash docker run -it --device=/dev/kfd --device=/dev/dri --group-add video horde-worker-regen:rocm ``` -------------------------------- ### Unregister Existing Ubuntu 24.04 WSL Image Source: https://github.com/haidra-org/horde-worker-regen/blob/main/README_advanced.md Resets the Ubuntu 24.04 WSL image to a clean state. This action will delete all data within the WSL image, so ensure any important data is backed up beforehand. ```bash wsl --unregister Ubuntu-24.04 ``` -------------------------------- ### Clone Horde Worker Repository Source: https://github.com/haidra-org/horde-worker-regen/blob/main/README_advanced.md Clone the worker repository to your local system using Git. ```bash git clone https://github.com/Haidra-Org/horde-worker-reGen.git ``` -------------------------------- ### Lower-end GPU Worker Settings Source: https://github.com/haidra-org/horde-worker-regen/blob/main/README.md Options for lower-end GPUs or under-performing workers. `extra_slow_worker` requires user opt-in and is for very low MPS/S or kudos/hr. ```yaml - extra_slow_worker: true - limit_max_steps: true - preload_timeout: 120 ``` -------------------------------- ### Build Argument to Disable Pip Cache Source: https://github.com/haidra-org/horde-worker-regen/blob/main/Dockerfiles/README.md Use this argument to troubleshoot pip cache-related errors by building the image without using the cache. ```bash --build-arg USE_PIP_CACHE=false ``` -------------------------------- ### Supported Stable Diffusion Baselines Source: https://github.com/haidra-org/horde-worker-regen/blob/main/README.md List of Stable Diffusion baseline categories currently supported for custom models. Flux.schnell series models are permitted, but Flux.dev models are not. ```text stable_diffusion_1 stable_diffusion_2_768 stable_diffusion_2_512 stable_diffusion_xl stable_cascade flux_1 ``` -------------------------------- ### Set Horde Worker Configuration via Environment Variables Source: https://github.com/haidra-org/horde-worker-regen/blob/main/Dockerfiles/README.md Configure the Horde worker by setting environment variables. Prepend 'AIWORKER_' to any option found in `bridgeData_template.yaml`. Ensure sensitive information like API keys and worker names are updated. ```shell AIWORKER_API_KEY=your_api_key_here # Important AIWORKER_CACHE_HOME=/workspace/models # Important AIWORKER_DREAMER_NAME=your_worker_name_here # Important AIWORKER_ALLOW_CONTROLNET=True AIWORKER_ALLOW_LORA=True AIWORKER_MAX_LORA_CACHE_SIZE=50 AIWORKER_ALLOW_PAINTING=True AIWORKER_MAX_POWER=38 AIWORKER_MAX_THREADS=1 # Only set to 2 on high end or xx90 machines AIWORKER_MODELS_TO_LOAD=['TOP 3', 'AlbedoBase XL (SDXL)'] # Be mindful of download times; each model average 2-8 gb AIWORKER_MODELS_TO_SKIP=['pix2pix', 'SDXL_beta::stability.ai#6901'] AIWORKER_QUEUE_SIZE=2 AIWORKER_MAX_BATCH=4 AIWORKER_SAFETY_ON_GPU=True AIWORKER_CIVITAI_API_TOKEN=your_token_here ``` -------------------------------- ### 8-10GB VRAM Worker Configuration Source: https://github.com/haidra-org/horde-worker-regen/blob/main/README.md Configuration for GPUs with 8-10GB VRAM (e.g., 2080, 3060, 4060). Disable post-processing and SDXL ControlNet if not needed. ```yaml - queue_size: 1 # <32GB RAM: 0, 32GB: 1, >32GB: 2 - safety_on_gpu: false - max_threads: 1 - max_power: 32 # No higher - max_batch: 4 # No higher - allow_post_processing: false # If using SDXL/Flux, else can be true - allow_sdxl_controlnet: false ``` -------------------------------- ### Build Argument for Testing Specific Git Branches Source: https://github.com/haidra-org/horde-worker-regen/blob/main/Dockerfiles/README.md Use this build argument to test new features from a specific Git branch before merging it into the main branch. ```bash --build-arg GIT_BRANCH=feature-new-model ``` -------------------------------- ### Reference Custom Model in bridgeData.yaml Source: https://github.com/haidra-org/horde-worker-regen/blob/main/README.md Define custom models by specifying their name, baseline, and local filepath in the bridgeData.yaml configuration file. Ensure the model name does not conflict with existing models. ```yaml custom_models: - name: My Custom Model baseline: stable_diffusion_xl filepath: /path/to/model/file.safetensors ``` -------------------------------- ### 12-16GB VRAM Worker Configuration Source: https://github.com/haidra-org/horde-worker-regen/blob/main/README.md Settings suitable for GPUs with 12-16GB VRAM (e.g., 3080 Ti, 4070 Ti, 4080). Consider disabling safety_on_gpu for Cascade/Flux. ```yaml - queue_size: 1 # <32GB RAM: 0, 32GB: 1, >32GB: 2 - safety_on_gpu: true # Consider false if using Cascade/Flux - moderate_performance_mode: true - unload_models_from_vram_often: false - max_threads: 1 - max_power: 50 - max_batch: 4 # Or higher ``` -------------------------------- ### Update Dependencies with CUDA Source: https://github.com/haidra-org/horde-worker-regen/blob/main/README_advanced.md Manually update Python dependencies, including PyTorch for CUDA, after pulling changes from the repository. This ensures you have the latest required packages. ```bash python -m pip install -r requirements.txt -U https://download.pytorch.org/whl/cu128 ``` -------------------------------- ### Update Dependencies with RoCM Source: https://github.com/haidra-org/horde-worker-regen/blob/main/README_advanced.md Manually update Python dependencies, including PyTorch for RoCM, after pulling changes from the repository. This ensures you have the latest required packages. ```bash python -m pip install -r requirements.txt -U https://download.pytorch.org/whl/rocm6.2 ``` -------------------------------- ### Update Worker with Docker Compose Source: https://github.com/haidra-org/horde-worker-regen/blob/main/Dockerfiles/README.md Update the horde-worker-reGen application by pulling the latest changes, rebuilding the Docker image, and recreating the container using Docker Compose. This process ensures you are running the most recent version. ```bash cd horde-worker-reGen git pull docker compose -f Dockerfiles/compose.[cuda|rocm].yaml build --pull docker compose -f Dockerfiles/compose.[cuda|rocm].yaml up -dV ``` -------------------------------- ### Stop a Running Container Source: https://github.com/haidra-org/horde-worker-regen/blob/main/Dockerfiles/README.md Stop a running horde-worker-reGen container named 'reGen'. It is recommended to set the worker to maintenance mode via the API before stopping to avoid job loss. ```bash docker stop reGen ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.