Inverse rendering aims to recover both 3D geometry and physically meaningful material properties from images, enabling applications such as relighting and novel view synthesis. Optimization-based methods achieve high fidelity but require costly per-scene fitting, while image-space learning-based approaches often suffer from multi-view inconsistencies and lack an explicit 3D representation for stable novel view rendering.
We present a feed-forward multi-view reconstruction framework for inverse rendering that directly predicts a structured 3D Gaussian representation with intrinsic material attributes. Each Gaussian primitive is parameterized by mean, normal, opacity, rotation, scale, albedo, metallic, and roughness, enabling a disentangled and physically grounded scene representation. Our model integrates priors from a material estimation network with a multi-view 3D reconstruction backbone, allowing joint prediction of geometry and reflectance parameters in a single forward pass.
Experiments on synthetic and real-world datasets demonstrate improved multi-view consistency compared to 2D baselines, accurate material recovery, and stable novel view rendering. Our representation further supports physically-based relighting and more faithful modeling of view-dependent effects compared to existing RGB-based feed-forward reconstruction methods.
Given N posed images, a single feed-forward network fθ({Ii}, {Pi})→(đť’˘, â„ł) jointly recovers scene geometry đť’˘ and intrinsic materials â„ł via a dual-branch design.
A Geometry branch (ResNet → multi-view geometry encoder → feature matching / cost volume) and an Intrinsic branch (DINOv2 → multi-view intrinsic translator) feed a shared set of decoding heads. The heads predict depth, Gaussian normals, rotation/scale/opacity, and material maps (albedo, metallic, roughness), which are unprojected into a 3D Gaussian scene and rendered differentiably. Training supervises all rendered properties with L1 + LPIPS, an affine-invariant depth loss, and a cosine normal loss.
Geometry branch
Frozen ResNet feature pyramid; a transformer geometry encoder does cross-view attention, and feature matching builds a depth-candidate cost volume C.
Intrinsic branch
DINOv2 ViT-L/14 encodes each view; a 36-block translator alternates intra- and inter-view attention to produce material features Fm.
Decoding
Six DPT / Point-Transformer heads predict depth, normals, rotation/scale/opacity, and albedo/metallic/roughness; unprojection lifts them into 3D Gaussians.
Rendering & loss
A single differentiable rasterization pass renders material, normal, and depth maps for supervision and novel views.
From sparse input views (2-4), InvSplat reconstructs a 3D Gaussian scene and renders smooth novel-view trajectories.
Input views
We compare predicted intrinsics against 2D image-space methods on 32-frame videos from the RE10K dataset. Our method reconstructs the scene from only the first and last frames of the camera path, then renders the full trajectory; the baselines instead run on every frame of the input video. Because our scene is a single 3D Gaussian field, its predictions stay multi-view consistent across the camera path, whereas the image-space baselines flicker on highlights and reflective surfaces.
Because each Gaussian carries physically-based materials and a surface normal, we can re-render the scene with arbitrary point light sources.
Input views
@inproceedings{invsplat2026,
title = {InvSplat: Inverse Feed-Forward Scene Splatting},
author = {Karpikova, Polina and Bian, Wenjing and Xu, Haofei and Lensch, Hendrik P. A. and Geiger, Andreas},
booktitle = {Arxiv},
year = {2026},
eprint = {2607.02301},
archivePrefix = {arXiv},
primaryClass = {cs.CV}
}