M^3: Dense Matching Meets Multi-View Foundation Models for Monocular Gaussian Splatting SLAM

3D scene reconstruction has become a fundamental capability in computer vision, enabling applications ranging from robotic perception to large-scale scene digitization. Recently, the field has been revolutionized by two paradigms: per-scene optimization, such as 3D Gaussian Splatting (3DGS), which delivers high-fidelity rendering, and feed-forward geometric foundation models, which infer dense priors in a single pass. However, most existing foundation models are inherently batch-oriented, designed to process a fixed set of images jointly. This offline nature precludes real-time feedback and limits scalability in open-ended environments, underscoring the urgent need for streaming reconstruction, where camera trajectories and scene geometry are incrementally updated as new observations arrive.

Existing efforts toward streaming 3D reconstruction generally follow two trajectories, yet both face significant hurdles. The first family attempts to adapt feed-forward models to a streaming context by incorporating memory mechanisms that summarize past observations to predict geometry incrementally. While these methods are efficient, they typically produce low-resolution results and struggle with cumulative drift, as they lack the iterative global refinement mechanisms found in classical SLAM. The second family instead integrates foundation-model priors into a SLAM pipeline to guide optimization. However, these approaches are often trapped in a fundamental trade-off: pairwise-prior methods, such as MASt3R-SLAM, suffer from redundant computation and quadratic complexity, whereas multi-frame prior methods like VGGT-SLAM 2.0 provide global geometry but lack the pixel-level dense correspondences necessary for rigorous geometric optimization.

M^3 bridges the gap by tightly coupling a multi-view foundation model with a robust SLAM pipeline. • It enhances a state-of-the-art multi-view geometric foundation model by introducing a dedicated dense matching head, specifically trained to recover pixel-level correspondences. • This enables the SLAM framework to leverage the foundation model’s geometry for accurate, high-frequency pose refinement. • Unlike previous black-box integrations, M^3 performs a single feed-forward inference over both historical keyframes and incoming frames to simultaneously update geometry and tracking, significantly reducing redundant model invocations. • A dynamic region identification module is introduced to detect and suppress transient objects, ensuring stable static scene reconstruction in real-world environments.

The M^3 method involves several key steps: • Enhancing a multi-view geometric foundation model with a dedicated dense matching head, trained to recover pixel-level correspondences. • Leveraging the foundation model’s geometry for accurate, high-frequency pose refinement within the SLAM framework. • Performing a single feed-forward inference over both historical keyframes and incoming frames to simultaneously update geometry and tracking, reducing redundant model invocations. • Introducing a dynamic region identification module to detect and suppress transient objects, ensuring stable static scene reconstruction in real-world environments. • Extensive experiments demonstrate state-of-the-art accuracy in both pose estimation and 3D reconstruction.

The experimental design includes testing on diverse indoor and outdoor datasets such as ScanNet++, ScanNetV2, Waymo, and KITTI. Baselines for comparison include DROID-SLAM, MASt3R-SLAM, VGGT-SLAM, VGGT-SLAM 2.0, and ARTDECO. Evaluation metrics include Absolute Trajectory Error (ATE) RMSE, PSNR, SSIM, and LPIPS. Key hyperparameters include the matching search radius and keyframe insertion threshold. Ablation studies are conducted to evaluate the contribution of individual components.

Experimental results show that M^3 achieves state-of-the-art accuracy in pose estimation and scene reconstruction. For example, on the ScanNet++ dataset, M^3 outperforms ARTDECO with a 2.11 dB improvement in PSNR, while reducing ATE RMSE by 64.3% compared to VGGT-SLAM 2.0. Ablation studies reveal that the matching head and dynamic region identification module are crucial for performance improvement. M^3 maintains competitive efficiency on long-duration monocular video streams.

The M^3 method can be directly applied to scenarios such as robotic navigation, augmented reality, and drone surveillance. These applications require high-precision pose estimation and scene reconstruction to enable real-time environmental perception and interaction. M^3's efficient computational performance makes it suitable for processing long-duration video streams, particularly in dynamic environments.

Despite its strong performance across various scenarios, M^3 may experience performance degradation in extremely dynamic scenes. Additionally, the method may be limited on devices with constrained computational resources, as multi-view processing and Gaussian splatting require high computational power. Future research directions include optimizing M^3's performance on resource-constrained devices and further enhancing its robustness in extremely dynamic scenes.