AdaptiveSplat:Texture Aware Controllable 3D Gaussian Allocation for Feed-Forward Reconstruction
AdaptiveSplat uses texture-aware pruning and adaptive Gaussian prediction to improve sparse 3D scene reconstruction without fine-tuning.
Key Findings
Methodology
The approach combines scene texture estimation via discrete wavelet transform (DWT), texture-aware pruning based on regional frequency energy, and an adaptive Gaussian head that re-predicts attributes of retained primitives. Texture estimation captures local scene complexity, guiding the selective removal of redundant Gaussians in low-frequency regions. The adaptive head then refines the attributes of remaining primitives, ensuring scene fidelity without requiring inference-time fine-tuning. This pipeline maintains the feed-forward paradigm, enabling controllable primitive allocation across various models.
Key Results
- On datasets like RE10K, ACID, DL3DV, Tanks and Temples, and DTU, AdaptiveSplat outperforms traditional pruning methods, achieving PSNR improvements up to 22.55dB (ACID) and 22.29dB (RE10K) at pruning ratios up to 80%. LPIPS scores are reduced to 0.299, indicating sharper, more detailed reconstructions. The method maintains high visual fidelity even at aggressive sparsity levels, demonstrating robustness across diverse scene complexities.
- In sparse-view scenarios with only two input images, the method produces high-quality reconstructions comparable to dense-view models, significantly reducing computational and storage costs. Extensive ablation studies confirm the importance of texture-guided pruning and the adaptive head, with ablation variants showing performance drops of over 3dB PSNR when these components are removed.
- The approach generalizes well across datasets, with minimal fine-tuning required, and achieves real-time inference speeds, making it suitable for practical applications such as VR/AR, industrial inspection, and rapid scene modeling.
Significance
This work addresses the longstanding challenge of balancing scene detail fidelity with model sparsity in feed-forward 3D reconstruction. By integrating texture analysis into the primitive selection process, it introduces a controllable, resource-efficient framework that adapts to scene complexity dynamically. The method reduces redundancy without sacrificing quality, paving the way for scalable, real-time 3D scene understanding. Its compatibility with existing architectures and elimination of inference-time fine-tuning mark significant progress toward practical deployment in interactive and resource-constrained environments.
Technical Contribution
The key innovations include: 1) a scene energy estimation module based on DWT for local frequency analysis; 2) a texture-aware pruning strategy that adaptively removes primitives in low-detail regions; 3) an adaptive Gaussian head that re-predicts attributes of retained primitives, ensuring scene consistency; 4) training with a stochastic pruning ratio to enhance robustness across different sparsity levels. These contributions collectively enable precise control over primitive density, improve reconstruction fidelity, and eliminate the need for post-processing fine-tuning, representing a fundamental advance in feed-forward 3D scene modeling.
Novelty
This is the first work to incorporate local texture information explicitly into the control of pixel-aligned 3D Gaussian primitives within a feed-forward pipeline. Unlike prior methods that rely solely on geometric heuristics or scene-specific optimization, this approach leverages frequency-based texture analysis to guide primitive pruning, combined with a novel adaptive attribute prediction mechanism. This synergy introduces a new paradigm for resource-aware, controllable 3D reconstruction, bridging the gap between dense, high-fidelity models and sparse, efficient representations.
Limitations
- The reliance on accurate texture estimation may limit performance in scenes with uniform textures or high noise levels, where frequency analysis becomes unreliable.
- While the adaptive head improves attribute prediction, residual errors can still cause minor artifacts or detail loss at very high compression ratios (β>0.8).
- The current framework primarily targets static scenes; extending to dynamic environments with moving objects or changing textures remains a challenge and requires further research.
Future Work
Future directions include integrating multi-scale texture analysis to better handle complex scenes, developing more robust frequency estimation methods in noisy conditions, and extending the framework to dynamic scenes with temporal consistency. Additionally, combining this approach with learned priors or neural implicit representations could further enhance scene fidelity and control. Exploring hardware acceleration and optimization for real-time deployment in AR/VR and robotics is also a promising avenue.
AI Executive Summary
Recent advances in feed-forward 3D scene reconstruction have enabled rapid scene modeling from limited views, yet these methods often produce highly redundant representations, especially in complex scenes. Traditional pixel-aligned Gaussian primitives, while efficient, lack explicit control over scene detail allocation, leading to over-smoothing or artifacts when aggressively pruned. Addressing this, AdaptiveSplat introduces a texture-aware framework that explicitly controls the number of Gaussian primitives based on local scene complexity.
The core idea hinges on estimating scene texture via discrete wavelet transform (DWT), which captures local frequency variations. Regions with high-frequency textures—rich in details—are preserved with more primitives, while smooth, low-frequency areas undergo aggressive pruning. This selective pruning is guided by a novel scene energy metric, enabling the model to dynamically allocate resources where needed most.
To compensate for the primitives removed, an adaptive Gaussian head is employed, which re-predicts the attributes of the remaining primitives, ensuring scene fidelity without the need for inference-time fine-tuning. This design maintains the feed-forward nature of the pipeline, allowing seamless integration with existing models.
Extensive experiments on datasets like RE10K, ACID, DL3DV, Tanks and Temples, and DTU demonstrate that AdaptiveSplat significantly outperforms baseline pruning strategies. It achieves higher PSNR and SSIM scores, with improvements up to 3dB PSNR and substantial reductions in perceptual error scores, even at high sparsity levels (β=0.8). The ablation studies confirm the importance of texture-guided pruning and adaptive attribute prediction in maintaining scene quality.
This approach offers a scalable, controllable, and efficient solution for real-time 3D scene reconstruction, with broad implications for virtual reality, robotics, and digital content creation. Future work aims to extend the framework to dynamic scenes and incorporate multi-scale texture analysis for even more robust performance, pushing the boundaries of real-time, resource-efficient 3D modeling.
Deep Dive
Abstract
Current feed-forward 3D reconstruction methods predict pixel aligned Gaussian primitives, resulting in highly redundant representations. A natural solution is to prune the redundant Gaussians, but naive pruning introduces severe artifacts and often requires inference time fine-tuning, breaking the feed-forward paradigm. Based on previous works, high frequency regions require more Gaussian primitives, while low frequency regions can be represented with significantly fewer primitives. Motivated by this, we propose a novel approach to explicitly control the number of Gaussians by leveraging local texture information. Our approach achieves this through three key components: (1) texture estimation to capture spatial variation in scene detail, (2) texture-aware pruning that removes redundant Gaussians from low frequency regions, and (3) an adaptive Gaussian head that predicts the modified attributes of the retained primitives without breaking the feed-forward paradigm. Experiments on RE10K, ACID, DL3DV, Tanks and Temples, and DTU demonstrate the effectiveness of our approach, while ablation studies validate the contributions of its key components.