Authors: Ye Fang, Zeyi Sun, Shangzhan Zhang, Tong Wu, Yinghao Xu, Pan Zhang, Jiaqi Wang, Gordon Wetzstein, Dahua Lin
Abstract: Diffusion models have demonstrated remarkable success in image generation and
editing, with recent advancements enabling albedo-preserving image relighting.
However, applying these models to video relighting remains challenging due to
the lack of paired video relighting datasets and the high demands for output
fidelity and temporal consistency, further complicated by the inherent
randomness of diffusion models. To address these challenges, we introduce
RelightVid, a flexible framework for video relighting that can accept
background video, text prompts, or environment maps as relighting conditions.
Trained on in-the-wild videos with carefully designed illumination
augmentations and rendered videos under extreme dynamic lighting, RelightVid
achieves arbitrary video relighting with high temporal consistency without
intrinsic decomposition while preserving the illumination priors of its image
backbone.
Source: http://arxiv.org/abs/2501.16330v1
