StructSplat: Generalizable 3D Gaussian Splatting from Uncalibrated Sparse Views

Recent advances in 3D scene reconstruction, exemplified by Neural Radiance Fields (NeRF), have revolutionized the field by enabling photorealistic novel view synthesis. However, these methods typically depend on precise camera calibration, which limits their deployment in real-world scenarios where such information is unavailable or costly to obtain. To address this, researchers have explored pose-free and intrinsic-free approaches, but many rely on scene-specific optimization or pre-estimated camera parameters, hindering scalability and generalization.

This paper introduces StructSplat, a novel framework that performs feed-forward, pose-free 3D Gaussian reconstruction directly from uncalibrated images. The core innovation lies in a structured representation that explicitly separates geometric, semantic, and texture cues, allowing the model to learn robust scene priors. The architecture employs a geometry encoder based on transformer models, a semantic encoder utilizing pre-trained vision models, and a lightweight texture encoder that injects pixel-aligned features. A key component is the camera alignment strategy, which estimates and aligns camera parameters across views without leakage, ensuring the model's generalization to unseen scenes.

Experimental results on benchmarks such as DL3DV demonstrate a PSNR of 28.045 dB, surpassing prior state-of-the-art methods like AnySplat by over 5 dB. Cross-dataset evaluations on ACID and RealEstate10K further confirm the model's robustness, with improvements of nearly 2 dB. These results highlight the method's ability to produce high-fidelity reconstructions without relying on camera calibration, significantly broadening the applicability of 3D scene understanding.

The implications of this work are profound, enabling scalable, real-time scene reconstruction in uncontrolled environments—crucial for virtual reality, autonomous navigation, and cultural heritage preservation. Despite these advances, challenges remain in handling extreme lighting, dynamic scenes, and reducing computational costs. Future research will focus on integrating additional cues, optimizing algorithms for speed, and extending to larger, more complex environments, promising a new era of flexible and accurate 3D modeling.