This ComfyUI workflow guides you through preparing a dataset, training a LoRA model, and using it in your generation pipeline. LoRA enables fine-tuning large models with just 10-50 images, creating consistent brand styles or character likenesses.
TL;DR — Key Takeaways
- Train Custom LoRA Models with ComfyUI — Advanced level
- Tool: ComfyUI
- 5 steps with detailed instructions
- Prerequisites: ComfyUI with Kohya training extension, GPU 12GB+ VRAM, 15-30 training images, SDXL or SD1.5 base model
Use Case
Brands wanting consistent visual style, artists creating character series, e-commerce with consistent product styling.
Prerequisites
ComfyUI with Kohya training extension, GPU 12GB+ VRAM, 15-30 training images, SDXL or SD1.5 base model
Workflow Diagram
Train Custom LoRA Models with ComfyUI
Step-by-Step Guide
-
Prepare Dataset
Collect 15-30 high-quality images of your subject/style. Crop to 512x512 or 1024x1024. Ensure variety in angles, lighting, and context. Create a text file for each image describing it. Place all in a training folder.
-
Configure Training Parameters
Use the Kohya LoRA training extension for ComfyUI. Set: base model (SDXL or SD1.5), learning rate 1e-4, batch size 1, epochs 10-20, LoRA rank 32-64. Higher rank = more style learning but more overfitting risk.
-
Start Training
Launch training. Monitor loss curves: training loss should decrease steadily, validation loss should not increase (overfitting signal). Training takes 1-4 hours depending on dataset size and GPU. Save checkpoints every 5 epochs.
-
Test the LoRA
Load the trained LoRA in ComfyUI using the LoRA Loader node. Generate images with prompts referencing your trained subject. Adjust LoRA strength (0.5-1.0) to balance style fidelity vs flexibility. Compare with base model outputs.
-
Integrate into Pipeline
Add the LoRA to your standard ComfyUI workflow. Use the LoRA Loader node before the generation node. Set strength to 0.7 as a starting point. Batch generate variations. Save successful prompts and settings for reuse.
Tags
This workflow answers:
- How to set up train custom lora models with comfyui
- What tools and access you need
- How to troubleshoot common issues