### Install and Set Up Modal Source: https://github.com/ostris/ai-toolkit/blob/main/README.md Install the Modal Python package and authenticate your Modal account. ```bash pip install modal modal setup ``` ```bash python -m modal setup ``` -------------------------------- ### Build and Start AI Toolkit UI Source: https://github.com/ostris/ai-toolkit/blob/main/README.md Navigates to the UI directory, builds the UI, and starts the development server. Access the UI via http://localhost:8675. ```bash cd ui npm run build_and_start ``` -------------------------------- ### Install and Run AI Toolkit (macOS) Source: https://github.com/ostris/ai-toolkit/blob/main/README.md Clones the repository, makes the run script executable, and runs the convenience script for macOS installation and UI startup. ```bash git clone https://github.com/ostris/ai-toolkit.git cd ai-toolkit chmod +x run_mac.zsh ./run_mac.zsh ``` -------------------------------- ### Build and Run UI Source: https://github.com/ostris/ai-toolkit/blob/main/dgx_instructions.md Navigates to the UI directory and executes the build and start script. ```bash cd ui npm run build_and_start ``` -------------------------------- ### Install AI Toolkit and Dependencies (Windows) Source: https://github.com/ostris/ai-toolkit/blob/main/README.md Clones the repository, sets up a virtual environment, activates it, installs PyTorch with CUDA support, and project dependencies for Windows. ```bash git clone https://github.com/ostris/ai-toolkit.git cd ai-toolkit python -m venv venv .\venv\Scripts\activate pip install --no-cache-dir torch==2.9.1 torchvision==0.24.1 torchaudio==2.9.1 --index-url https://download.pytorch.org/whl/cu128 pip install -r requirements.txt ``` -------------------------------- ### Install PyTorch and Requirements (Linux/macOS) Source: https://github.com/ostris/ai-toolkit/blob/main/README.md Installs PyTorch with CUDA support and project dependencies. Ensure PyTorch is installed first. ```bash pip3 install --no-cache-dir torch==2.9.1 torchvision==0.24.1 torchaudio==2.9.1 --index-url https://download.pytorch.org/whl/cu128 pip3 install -r requirements.txt ``` -------------------------------- ### Initialize Submodules and Install Requirements Source: https://github.com/ostris/ai-toolkit/blob/main/notebooks/FLUX_1_dev_LoRA_Training.ipynb Navigates into the cloned AI toolkit directory, initializes its Git submodules, and installs all necessary Python dependencies listed in `requirements.txt`. This step is crucial for the toolkit to function correctly. ```bash !cd ai-toolkit && git submodule update --init --recursive && pip install -r requirements.txt ``` -------------------------------- ### Install PyTorch and Requirements Source: https://github.com/ostris/ai-toolkit/blob/main/dgx_instructions.md Installs PyTorch dependencies and project-specific requirements via pip. ```bash pip3 install torch==2.9.1 torchvision==0.24.1 torchaudio==2.9.1 --index-url https://download.pytorch.org/whl/cu130 ``` ```bash pip3 install -r dgx_requirements.txt ``` -------------------------------- ### Install Miniconda on Linux ARM64 Source: https://github.com/ostris/ai-toolkit/blob/main/dgx_instructions.md Downloads and executes the Miniconda installer for the DGX OS architecture. ```bash wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-aarch64.sh chmod u+x Miniconda3-latest-Linux-aarch64.sh ./Miniconda3-latest-Linux-aarch64.sh ``` -------------------------------- ### Start Gradio UI for Training Source: https://github.com/ostris/ai-toolkit/blob/main/README.md Logs into Hugging Face with a write token and starts the Gradio UI for image uploading, captioning, training, and publishing LoRAs. ```bash cd ai-toolkit #in case you are not yet in the ai-toolkit folder huggingface-cli login #provide a `write` token to publish your LoRA at the end python flux_train_ui.py ``` -------------------------------- ### Clone and Set Up ai-toolkit Repository Source: https://github.com/ostris/ai-toolkit/blob/main/README.md Clone the ai-toolkit repository, update submodules, create and activate a virtual environment, and install core dependencies. ```bash git clone https://github.com/ostris/ai-toolkit.git cd ai-toolkit git submodule update --init --recursive python -m venv venv source venv/bin/activate pip install torch pip install -r requirements.txt pip install --upgrade accelerate transformers diffusers huggingface_hub ``` -------------------------------- ### Run Next.js Development Server Source: https://github.com/ostris/ai-toolkit/blob/main/ui/README.md Use these commands to start the development server for your Next.js project. Open http://localhost:3000 in your browser to view the result. ```bash npm run dev # or yarn dev # or pnpm dev # or bun dev ``` -------------------------------- ### Clone and Set Up AI Toolkit Environment Source: https://github.com/ostris/ai-toolkit/blob/main/README.md This snippet shows how to clone the AI Toolkit repository and set up a Python virtual environment on Linux. Ensure you have Python 3.10+ and Git installed. ```bash git clone https://github.com/ostris/ai-toolkit.git cd ai-toolkit python3 -m venv venv source venv/bin/activate ``` -------------------------------- ### Secure AI Toolkit UI with Auth Token (Linux) Source: https://github.com/ostris/ai-toolkit/blob/main/README.md Sets the AI_TOOLKIT_AUTH environment variable to secure the UI and then starts the UI build process. ```bash # Linux AI_TOOLKIT_AUTH=super_secure_password npm run build_and_start ``` -------------------------------- ### Secure AI Toolkit UI with Auth Token (Windows CMD) Source: https://github.com/ostris/ai-toolkit/blob/main/README.md Sets the AI_TOOLKIT_AUTH environment variable using the command prompt to secure the UI and then starts the UI build process. ```bash # Windows set AI_TOOLKIT_AUTH=super_secure_password && npm run build_and_start ``` -------------------------------- ### Secure AI Toolkit UI with Auth Token (Windows PowerShell) Source: https://github.com/ostris/ai-toolkit/blob/main/README.md Sets the AI_TOOLKIT_AUTH environment variable using PowerShell to secure the UI and then starts the UI build process. ```bash # Windows Powershell $env:AI_TOOLKIT_AUTH="super_secure_password"; npm run build_and_start ``` -------------------------------- ### Import AI Toolkit Components and Configure Hub Transfer Source: https://github.com/ostris/ai-toolkit/blob/main/notebooks/FLUX_1_dev_LoRA_Training.ipynb Imports necessary modules from the AI toolkit, including `run_job`, and configures Hugging Face Hub transfer settings. This setup is essential for running training jobs and efficient model downloads. ```python import os import sys sys.path.append('/content/ai-toolkit') from toolkit.job import run_job from collections import OrderedDict from PIL import Image import os os.environ["HF_HUB_ENABLE_HF_TRANSFER"] = "1" ``` -------------------------------- ### Run Training with Modal Source: https://github.com/ostris/ai-toolkit/blob/main/README.md Execute the training process by running the Modal script with a specified configuration file. ```bash modal run run_modal.py --config-file-list-str=/root/ai-toolkit/config/whatever_you_want.yml ``` -------------------------------- ### Configure Node.js Path Source: https://github.com/ostris/ai-toolkit/blob/main/dgx_instructions.md Adds the extracted Node.js binary directory to the system PATH in .bashrc. ```bash export PATH=“/opt/node-v24.11.1-linux-arm64/bin:$PATH” ``` -------------------------------- ### Clone AI Toolkit Repository Source: https://github.com/ostris/ai-toolkit/blob/main/notebooks/SliderTraining.ipynb Clones the AI Toolkit repository from GitHub. This is the first step to set up the environment. ```bash !git clone https://github.com/ostris/ai-toolkit ``` -------------------------------- ### Run Training Job Configuration Source: https://github.com/ostris/ai-toolkit/blob/main/README.md Executes a training job using a specified YAML configuration file. Ensure the config file is edited and placed correctly. ```bash python run.py config/whatever_you_want.yml ``` -------------------------------- ### Configure Local Directory Mount for Modal Source: https://github.com/ostris/ai-toolkit/blob/main/README.md Set the local path to your ai-toolkit directory in the `run_modal.py` script for code mounting. ```python code_mount = modal.Mount.from_local_dir("/Users/username/ai-toolkit", remote_path="/root/ai-toolkit") ``` -------------------------------- ### Configure AI Prompts and Parameters Source: https://github.com/ostris/ai-toolkit/blob/main/notebooks/FLUX_1_schnell_LoRA_Training.ipynb Defines a list of prompts and associated parameters for an AI job. Use this to set up negative prompts, seeds, guidance scale, and sampling steps. ```python job_to_run = ( 'job_name', 'output/LoRA/your_name_v1', # output dir 'output/LoRA/your_name_v1/samples', # sample dir [ # list of jobs ('lora_img2img', [ ('prompt', [ 'you can add [trigger] to the prompts here and it will be replaced with the trigger word', 'a woman with red hair, playing chess at the park, bomb going off in the background', 'a woman holding a coffee cup, in a beanie, sitting at a cafe', 'a horse is a DJ at a night club, fish eye lens, smoke machine, lazer lights, holding a martini', 'a man showing off his cool new t shirt at the beach, a shark is jumping out of the water in the background', 'a bear building a log cabin in the snow covered mountains', 'woman playing the guitar, on stage, singing a song, laser lights, punk rocker', 'hipster man with a beard, building a chair, in a wood shop', 'photo of a man, white background, medium shot, modeling clothing, studio lighting, white backdrop', 'a man holding a sign that says, \'this is a sign\'', 'a bulldog, in a post apocalyptic world, with a shotgun, in a leather jacket, in a desert, with a motorcycle' ]), ('neg', ''), # not used on flux ('seed', 42), ('walk_seed', True), ('guidance_scale', 1), # schnell does not do guidance ('sample_steps', 4) # 1 - 4 works well ]) ]), # you can add any additional meta info here. [name] is replaced with config name at top ('meta', OrderedDict([ ('name', '[name]'), ('version', '1.0') ])) ]) ``` -------------------------------- ### Log in to Hugging Face CLI Source: https://github.com/ostris/ai-toolkit/blob/main/README.md Log in to the Hugging Face CLI using your read token to access models and datasets. ```bash huggingface-cli login ``` -------------------------------- ### Clone AI Toolkit and Prepare Dataset Directory Source: https://github.com/ostris/ai-toolkit/blob/main/notebooks/FLUX_1_dev_LoRA_Training.ipynb Clones the Ostris AI Toolkit repository from GitHub and creates a directory for storing image datasets. Ensure the dataset is placed in the `/content/dataset` folder. ```bash !git clone https://github.com/ostris/ai-toolkit !mkdir -p /content/dataset ``` -------------------------------- ### Configure Training Job with OrderedDict Source: https://github.com/ostris/ai-toolkit/blob/main/notebooks/SliderTraining.ipynb Defines the training parameters, model source, and saving behavior for a LoRA slider job. Ensure the training_folder path is correctly set for your environment. ```python from collections import OrderedDict job_to_run = OrderedDict({ # This is the config I use on my sliders, It is solid and tested 'job': 'train', 'config': { # the name will be used to create a folder in the output folder # it will also replace any [name] token in the rest of this config 'name': 'detail_slider_v1', # folder will be created with name above in folder below # it can be relative to the project root or absolute 'training_folder': "output/LoRA", 'device': 'cuda', # cpu, cuda:0, etc # for tensorboard logging, we will make a subfolder for this job 'log_dir': "output/.tensorboard", # you can stack processes for other jobs, It is not tested with sliders though # just use one for now 'process': [ { 'type': 'slider', # tells runner to run the slider process # network is the LoRA network for a slider, I recommend to leave this be 'network': { 'type': "lora", # rank / dim of the network. Bigger is not always better. Especially for sliders. 8 is good 'linear': 8, # "rank" or "dim" 'linear_alpha': 4, # Do about half of rank "alpha" # 'conv': 4, # for convolutional layers "locon" # 'conv_alpha': 4, # Do about half of conv "alpha" }, # training config 'train': { # this is also used in sampling. Stick with ddpm unless you know what you are doing 'noise_scheduler': "ddpm", # or "ddpm", "lms", "euler_a" # how many steps to train. More is not always better. I rarely go over 1000 'steps': 100, # I have had good results with 4e-4 to 1e-4 at 500 steps 'lr': 2e-4, # enables gradient checkpoint, saves vram, leave it on 'gradient_checkpointing': True, # train the unet. I recommend leaving this true 'train_unet': True, # train the text encoder. I don't recommend this unless you have a special use case # for sliders we are adjusting representation of the concept (unet), # not the description of it (text encoder) 'train_text_encoder': False, # just leave unless you know what you are doing # also supports "dadaptation" but set lr to 1 if you use that, # but it learns too fast and I don't recommend it 'optimizer': "adamw", # only constant for now 'lr_scheduler': "constant", # we randomly denoise random num of steps form 1 to this number # while training. Just leave it 'max_denoising_steps': 40, # works great at 1. I do 1 even with my 4090. # higher may not work right with newer single batch stacking code anyway 'batch_size': 1, # bf16 works best if your GPU supports it (modern) 'dtype': 'bf16', # fp32, bf16, fp16 # I don't recommend using unless you are trying to make a darker lora. Then do 0.1 MAX # although, the way we train sliders is comparative, so it probably won't work anyway 'noise_offset': 0.0, }, # the model to train the LoRA network on 'model': { # name_or_path can be a hugging face name, local path or url to model # on civit ai with or without modelVersionId. They will be cached in /model folder # epicRealisim v5 'name_or_path': "https://civitai.com/models/25694?modelVersionId=134065", 'is_v2': False, # for v2 models 'is_v_pred': False, # for v-prediction models (most v2 models) # has some issues with the dual text encoder and the way we train sliders # it works bit weights need to probably be higher to see it. 'is_xl': False, # for SDXL models }, # saving config 'save': { 'dtype': 'float16', # precision to save. I recommend float16 'save_every': 50, # save every this many steps # this will remove step counts more than this number ``` -------------------------------- ### Download Model from Modal Volume Source: https://github.com/ostris/ai-toolkit/blob/main/README.md Download a specific model from the Modal volume to your local machine. ```bash modal volume get flux-lora-models your-model-name ``` ```bash modal volume get flux-lora-models my_first_flux_lora_v1 ``` -------------------------------- ### Run Training Script Manually Source: https://github.com/ostris/ai-toolkit/blob/main/dgx_instructions.md Executes the training process directly using a configuration file for debugging purposes. ```bash python run.py path/to/train.yaml ``` -------------------------------- ### Configure Flux LoRA Training Job Source: https://github.com/ostris/ai-toolkit/blob/main/notebooks/FLUX_1_dev_LoRA_Training.ipynb Defines the training job parameters, including dataset paths, model selection, and optimization settings. Requires the collections.OrderedDict class to maintain configuration order. ```python from collections import OrderedDict job_to_run = OrderedDict([ ('job', 'extension'), ('config', OrderedDict([ # this name will be the folder and filename name ('name', 'my_first_flux_lora_v1'), ('process', [ OrderedDict([ ('type', 'sd_trainer'), # root folder to save training sessions/samples/weights ('training_folder', '/content/output'), # uncomment to see performance stats in the terminal every N steps #('performance_log_every', 1000), ('device', 'cuda:0'), # if a trigger word is specified, it will be added to captions of training data if it does not already exist # alternatively, in your captions you can add [trigger] and it will be replaced with the trigger word # ('trigger_word', 'image'), ('network', OrderedDict([ ('type', 'lora'), ('linear', 16), ('linear_alpha', 16) ])), ('save', OrderedDict([ ('dtype', 'float16'), # precision to save ('save_every', 250), # save every this many steps ('max_step_saves_to_keep', 4) # how many intermittent saves to keep ])), ('datasets', [ # datasets are a folder of images. captions need to be txt files with the same name as the image # for instance image2.jpg and image2.txt. Only jpg, jpeg, and png are supported currently # images will automatically be resized and bucketed into the resolution specified OrderedDict([ ('folder_path', '/content/dataset'), ('caption_ext', 'txt'), ('caption_dropout_rate', 0.05), # will drop out the caption 5% of time ('shuffle_tokens', False), # shuffle caption order, split by commas ('cache_latents_to_disk', True), # leave this true unless you know what you're doing ('resolution', [512, 768, 1024]) # flux enjoys multiple resolutions ]) ]), ('train', OrderedDict([ ('batch_size', 1), ('steps', 2000), # total number of steps to train 500 - 4000 is a good range ('gradient_accumulation_steps', 1), ('train_unet', True), ('train_text_encoder', False), # probably won't work with flux ('content_or_style', 'balanced'), # content, style, balanced ('gradient_checkpointing', True), # need the on unless you have a ton of vram ('noise_scheduler', 'flowmatch'), # for training only ('optimizer', 'adamw8bit'), ('lr', 1e-4), # uncomment this to skip the pre training sample # ('skip_first_sample', True), # uncomment to completely disable sampling # ('disable_sampling', True), # uncomment to use new vell curved weighting. Experimental but may produce better results # ('linear_timesteps', True), # ema will smooth out learning, but could slow it down. Recommended to leave on. ('ema_config', OrderedDict([ ('use_ema', True), ('ema_decay', 0.99) ])), # will probably need this if gpu supports it for flux, other dtypes may not work correctly ('dtype', 'bf16') ])), ('model', OrderedDict([ # huggingface model name or path ('name_or_path', 'black-forest-labs/FLUX.1-dev'), ('is_flux', True), ('quantize', True), # run 8bit mixed precision #('low_vram', True), # uncomment this if the GPU is connected to your monitors. It will use less vram to quantize, but is slower. ])), ('sample', OrderedDict([ ('sampler', 'flowmatch'), # must match train.noise_scheduler ('sample_every', 250), # sample every this many steps ('width', 1024), ('height', 1024), ('prompts', [ # you can add [trigger] to the prompts here and it will be replaced with the trigger word #'[trigger] holding a sign that says 'I LOVE PROMPTS!'', ``` -------------------------------- ### Execute AI Job Source: https://github.com/ostris/ai-toolkit/blob/main/notebooks/FLUX_1_schnell_LoRA_Training.ipynb Runs the configured AI job. Ensure the job configuration is complete before execution. Output will be saved to the specified directories. ```python run_job(job_to_run) ``` -------------------------------- ### Configure Conda Environment Source: https://github.com/ostris/ai-toolkit/blob/main/dgx_instructions.md Commands to manage the base environment activation and create a dedicated Python 3.11 environment. ```bash conda config --set auto_activate_base false ``` ```bash conda create --name ai-toolkit python=3.11 ``` ```bash conda activate ai-toolkit ``` -------------------------------- ### AI Toolkit Configuration Source: https://github.com/ostris/ai-toolkit/blob/main/notebooks/FLUX_1_schnell_LoRA_Training.ipynb This configuration defines a training job for a LoRA model using the FLUX.1-schnell architecture. It specifies training parameters, dataset paths, model settings, and sampling configurations. Ensure dataset images and captions are correctly formatted. ```python from collections import OrderedDict job_to_run = OrderedDict([ ('job', 'extension'), ('config', OrderedDict([ # this name will be the folder and filename name ('name', 'my_first_flux_lora_v1'), ('process', [ OrderedDict([ ('type', 'sd_trainer'), # root folder to save training sessions/samples/weights ('training_folder', '/content/output'), # uncomment to see performance stats in the terminal every N steps #('performance_log_every', 1000), ('device', 'cuda:0'), # if a trigger word is specified, it will be added to captions of training data if it does not already exist # alternatively, in your captions you can add [trigger] and it will be replaced with the trigger word # ('trigger_word', 'image'), ('network', OrderedDict([ ('type', 'lora'), ('linear', 16), ('linear_alpha', 16) ])), ('save', OrderedDict([ ('dtype', 'float16'), # precision to save ('save_every', 250), # save every this many steps ('max_step_saves_to_keep', 4) # how many intermittent saves to keep ])), ('datasets', [ # datasets are a folder of images. captions need to be txt files with the same name as the image # for instance image2.jpg and image2.txt. Only jpg, jpeg, and png are supported currently # images will automatically be resized and bucketed into the resolution specified OrderedDict([ ('folder_path', '/content/dataset'), ('caption_ext', 'txt'), ('caption_dropout_rate', 0.05), # will drop out the caption 5% of time ('shuffle_tokens', False), # shuffle caption order, split by commas ('cache_latents_to_disk', True), # leave this true unless you know what you're doing ('resolution', [512, 768, 1024]) # flux enjoys multiple resolutions ]) ]), ('train', OrderedDict([ ('batch_size', 1), ('steps', 2000), # total number of steps to train 500 - 4000 is a good range ('gradient_accumulation_steps', 1), ('train_unet', True), ('train_text_encoder', False), # probably won't work with flux ('gradient_checkpointing', True), # need the on unless you have a ton of vram ('noise_scheduler', 'flowmatch'), # for training only ('optimizer', 'adamw8bit'), ('lr', 1e-4), # uncomment this to skip the pre training sample # ('skip_first_sample', True), # uncomment to completely disable sampling # ('disable_sampling', True), # uncomment to use new vell curved weighting. Experimental but may produce better results # ('linear_timesteps', True), # ema will smooth out learning, but could slow it down. Recommended to leave on. ('ema_config', OrderedDict([ ('use_ema', True), ('ema_decay', 0.99) ])), # will probably need this if gpu supports it for flux, other dtypes may not work correctly ('dtype', 'bf16') ])), ('model', OrderedDict([ # huggingface model name or path ('name_or_path', 'black-forest-labs/FLUX.1-schnell'), ('assistant_lora_path', 'ostris/FLUX.1-schnell-training-adapter'), # Required for flux schnell training ('is_flux', True), ('quantize', True), # run 8bit mixed precision # low_vram is painfully slow to fuse in the adapter avoid it unless absolutely necessary #('low_vram', True), # uncomment this if the GPU is connected to your monitors. It will use less vram to quantize, but is slower. ])), ('sample', OrderedDict([ ('sampler', 'flowmatch'), # must match train.noise_scheduler ('sample_every', 250), # sample every this many steps ('width', 1024), ('height', 1024), ('prompts', [ ``` -------------------------------- ### Execute AI Generation Job Source: https://github.com/ostris/ai-toolkit/blob/main/notebooks/FLUX_1_dev_LoRA_Training.ipynb Runs the configured AI generation job. Ensure the 'run_job' function is available in your environment. ```python run_job(job_to_run) ``` -------------------------------- ### Configure Training Targets and Metadata Source: https://github.com/ostris/ai-toolkit/blob/main/notebooks/SliderTraining.ipynb Defines the training parameters for sliders, including target classes and prompt polarities, and sets model metadata. ```python 'batch_full_slide': True, # These are the concepts to train on. You can do as many as you want here, # but they can conflict outweigh each other. Other than experimenting, I recommend # just doing one for good results 'targets': [ # target_class is the base concept we are adjusting the representation of # for example, if we are adjusting the representation of a person, we would use "person" # if we are adjusting the representation of a cat, we would use "cat" It is not # a keyword necessarily but what the model understands the concept to represent. # "person" will affect men, women, children, etc but will not affect cats, dogs, etc # it is the models base general understanding of the concept and everything it represents # you can leave it blank to affect everything. In this example, we are adjusting # detail, so we will leave it blank to affect everything { 'target_class': "", # positive is the prompt for the positive side of the slider. # It is the concept that will be excited and amplified in the model when we slide the slider # to the positive side and forgotten / inverted when we slide # the slider to the negative side. It is generally best to include the target_class in # the prompt. You want it to be the extreme of what you want to train on. For example, # if you want to train on fat people, you would use "an extremely fat, morbidly obese person" # as the prompt. Not just "fat person" # max 75 tokens for now 'positive': "high detail, 8k, intricate, detailed, high resolution, high res, high quality", # negative is the prompt for the negative side of the slider and works the same as positive # it does not necessarily work the same as a negative prompt when generating images # these need to be polar opposites. # max 76 tokens for now 'negative': "blurry, boring, fuzzy, low detail, low resolution, low res, low quality", # the loss for this target is multiplied by this number. # if you are doing more than one target it may be good to set less important ones # to a lower number like 0.1 so they don't outweigh the primary target 'weight': 1.0, }, ], }, }, ] }, # You can put any information you want here, and it will be saved in the model. # The below is an example, but you can put your grocery list in it if you want. # It is saved in the model so be aware of that. The software will include this # plus some other information for you automatically 'meta': { # [name] gets replaced with the name above 'name': "[name]", 'version': '1.0', # 'creator': { # 'name': 'your name', # 'email': 'your@gmail.com', # 'website': 'https://your.website' # } } }) ``` -------------------------------- ### List Contents of Modal Volume Source: https://github.com/ostris/ai-toolkit/blob/main/README.md Check the contents of the Modal volume where models, samples, and optimizers are stored. ```bash modal volume ls flux-lora-models ``` -------------------------------- ### Check GPU Status Source: https://github.com/ostris/ai-toolkit/blob/main/notebooks/FLUX_1_dev_LoRA_Training.ipynb Verifies the availability and status of NVIDIA GPUs. This is a prerequisite for training on GPU-enabled environments. ```bash !nvidia-smi ``` -------------------------------- ### LoKr Training Configuration Source: https://github.com/ostris/ai-toolkit/blob/main/README.md Configure the network type to 'lokr' for LoKr training, specifying full rank and factor. ```yaml network: type: "lokr" lokr_full_rank: true lokr_factor: 8 ``` -------------------------------- ### AI Generation Job Configuration Source: https://github.com/ostris/ai-toolkit/blob/main/notebooks/FLUX_1_dev_LoRA_Training.ipynb Defines parameters for an AI generation job, including prompts, negative prompts, and sampling settings. Use this to customize image generation. ```python job_to_run = [ ('prompts', [ 'woman with red hair, playing chess at the park, bomb going off in the background', 'a woman holding a coffee cup, in a beanie, sitting at a cafe', 'a horse is a DJ at a night club, fish eye lens, smoke machine, lazer lights, holding a martini', 'a man showing off his cool new t shirt at the beach, a shark is jumping out of the water in the background', 'a bear building a log cabin in the snow covered mountains', 'woman playing the guitar, on stage, singing a song, laser lights, punk rocker', 'hipster man with a beard, building a chair, in a wood shop', 'photo of a man, white background, medium shot, modeling clothing, studio lighting, white backdrop', 'a man holding a sign that says, \'this is a sign\'', 'a bulldog, in a post apocalyptic world, with a shotgun, in a leather jacket, in a desert, with a motorcycle' ]), ('neg', ''), # not used on flux ('seed', 42), ('walk_seed', True), ('guidance_scale', 4), ('sample_steps', 20) ] ``` ```python # you can add any additional meta info here. [name] is replaced with config name at top ('meta', OrderedDict([ ('name', '[name]'), ('version', '1.0') ])) ``` -------------------------------- ### Set Hugging Face Token Environment Variable Source: https://github.com/ostris/ai-toolkit/blob/main/notebooks/FLUX_1_dev_LoRA_Training.ipynb Prompts the user to enter their Hugging Face access token and sets it as an environment variable `HF_TOKEN`. This is required for accessing gated models like FLUX.1-dev. Ensure you have accepted the model license on Hugging Face. ```python import getpass import os # Prompt for the token hf_token = getpass.getpass('Enter your HF access token and press enter: ') # Set the environment variable os.environ['HF_TOKEN'] = hf_token print("HF_TOKEN environment variable has been set.") ``` -------------------------------- ### Import AI Toolkit Modules Source: https://github.com/ostris/ai-toolkit/blob/main/notebooks/SliderTraining.ipynb Imports necessary modules from the AI Toolkit and standard Python libraries. This code is required for subsequent operations within the toolkit. ```python import os import sys sys.path.append('/content/ai-toolkit') from toolkit.job import run_job from collections import OrderedDict from PIL import Image ``` -------------------------------- ### LoRA Training Configuration: Transformer Blocks Source: https://github.com/ostris/ai-toolkit/blob/main/README.md Configure LoRA training to target all single transformer blocks within the model. ```yaml network: type: "lora" linear: 128 linear_alpha: 128 network_kwargs: only_if_contains: - "transformer.single_transformer_blocks." ``` -------------------------------- ### LoRA Training Configuration: Specific Layers Source: https://github.com/ostris/ai-toolkit/blob/main/README.md Configure LoRA training to target specific layers within the model by using `only_if_contains`. ```yaml network: type: "lora" linear: 128 linear_alpha: 128 network_kwargs: only_if_contains: - "transformer.single_transformer_blocks.7.proj_out" - "transformer.single_transformer_blocks.20.proj_out" ``` -------------------------------- ### LoRA Training Configuration: Exclude Layers Source: https://github.com/ostris/ai-toolkit/blob/main/README.md Configure LoRA training to exclude specific layers, such as all single transformer blocks, using `ignore_if_contains`. ```yaml network: type: "lora" linear: 128 linear_alpha: 128 network_kwargs: ignore_if_contains: - "transformer.single_transformer_blocks." ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.