Authors: Xi Liu, Chaoyi Zhou, Siyu Huang
Abstract: Novel-view synthesis aims to generate novel views of a scene from multiple
input images or videos, and recent advancements like 3D Gaussian splatting
(3DGS) have achieved notable success in producing photorealistic renderings
with efficient pipelines. However, generating high-quality novel views under
challenging settings, such as sparse input views, remains difficult due to
insufficient information in under-sampled areas, often resulting in noticeable
artifacts. This paper presents 3DGS-Enhancer, a novel pipeline for enhancing
the representation quality of 3DGS representations. We leverage 2D video
diffusion priors to address the challenging 3D view consistency problem,
reformulating it as achieving temporal consistency within a video generation
process. 3DGS-Enhancer restores view-consistent latent features of rendered
novel views and integrates them with the input views through a spatial-temporal
decoder. The enhanced views are then used to fine-tune the initial 3DGS model,
significantly improving its rendering performance. Extensive experiments on
large-scale datasets of unbounded scenes demonstrate that 3DGS-Enhancer yields
superior reconstruction performance and high-fidelity rendering results
compared to state-of-the-art methods. The project webpage is
https://xiliu8006.github.io/3DGS-Enhancer-project .
Source: http://arxiv.org/abs/2410.16266v1
