Dreambooth training with Diffusers: hyperparameters, schedulers, and hardware for Stable Diffusion fine-tuning

AI Impact Summary

Diffusers' Dreambooth script enables targeted fine-tuning of Stable Diffusion to embed new subjects or styles. The guidance highlights a practical hyperparameter sweet spot: low learning rate (2e-6–1e-6), 400–1200 steps, with prior preservation for faces, and optional text encoder fine-tuning for quality, plus memory optimizations like 8-bit Adam or fp16. It also notes scheduler choices (DDIM often yields better results when overfitting is a risk) and the typical multi-GPU setup (train_dreambooth.py with AdamW on 2x40 GB A100s). Ensure license compliance under CreativeML Open RAIL-M when distributing trained models and plan for compute and data governance for production use.