Authors: Tejas Jayashankar, J. Jon Ryu, Gregory Wornell
Abstract: We propose Score-of-Mixture Training (SMT), a novel framework for training
one-step generative models by minimizing a class of divergences called the
$\alpha$-skew Jensen-Shannon divergence. At its core, SMT estimates the score
of mixture distributions between real and fake samples across multiple noise
levels. Similar to consistency models, our approach supports both training from
scratch (SMT) and distillation using a pretrained diffusion model, which we
call Score-of-Mixture Distillation (SMD). It is simple to implement, requires
minimal hyperparameter tuning, and ensures stable training. Experiments on
CIFAR-10 and ImageNet 64×64 show that SMT/SMD are competitive with and can even
outperform existing methods.
Source: http://arxiv.org/abs/2502.09609v1
