SVG-IR: Spatially-varying Gaussian Splatting for Inverse Rendering

Hanxiao Sun1, Yupeng Gao2, Jin Xie2, Jian Yang1, Beibei Wang*2,
1Nankai University 2Nanjing University
CVPR 2025
Paper Supplementary Code arXiv

SVG-IR introduces a new spatially-varying Gaussian representation inspired by replacing flat shading for interpolated shading in triangle rendering. Each spatially-varying Gaussian allows spatially-varying material and normal attributes by interpolating among Gaussian vertices defined in the tangent space of a Gaussian. Compared to the original Gaussian (i.e., Constant Gaussian) with constant attributes, SVG has a more powerful representation ability to produce high-quality NVS and relighting results.

Abstract

Reconstructing 3D assets from images, known as inverse rendering (IR), remains a challenging task due to its ill-posed nature. 3D Gaussian Splatting (3DGS) has demonstrated impressive capabilities for novel view synthesis (NVS) tasks. Methods apply it to relighting by separating radiance into BRDF parameters and lighting, yet produce inferior relighting quality with artifacts and unnatural indirect illumination due to the limited capability of each Gaussian, which has constant material parameters and normal, alongside the absence of physical constraints for indirect lighting. In this paper, we present a novel framework called Spatially-vayring Gaussian Inverse Rendering (SVG-IR), aimed at enhancing both NVS and relighting quality. To this end, we propose a new representation—Spatially-varying Gaussian (SVG)—that allows per-Gaussian spatially varying parameters. This enhanced representation is complemented by a SVG splatting scheme akin to vertex/fragment shading in traditional graphics pipelines. Furthermore, we integrate a physically-based indirect lighting model, enabling more realistic relighting. The proposed SVG-IR framework significantly improves rendering quality, outperforming state-of-the-art NeRF-based methods by 2.5 dB in peak signal-to-noise ratio (PSNR) and surpassing existing Gaussian-based techniques by 3.5 dB in relighting tasks, all while maintaining a real-time rendering speed.

Pipeline

MY ALT TEXT

Overview of our framework. We propose a novel SVG-IR framework. Within this framework, we introduce a Spatially-varying Gaussian representation capable of spatially variability with material attributes. We employ SVG splatting, analogous to vertex and fragment shading in the traditional triangle rendering pipeline, to leverage the improved appearance capability of SVGs. Additionally, we present a physics-based lighting model that enforces additional physical constraints to facilitate the decoupling of lighting and material properties.

Results

BibTeX

        @article{
        author={Hanxiao Sun and Yupeng Gao and Jin Xie and Jian Yang and Beibei Wang},
        title={SVG-IR: Spatially-varying Gaussian Splatting for Inverse Rendering},
        year={2025},
        journal={Proceedings of CVPR 2025},
      }