Benchmarking Robustness of Endoscopic Depth Estimation with Synthetically Corrupted Data

The evolution of endoscopic surgery has significantly improved patient outcomes by enabling minimally invasive procedures. Early methods relied on optical cues and stereo vision, but these approaches faced limitations due to cost and complexity. Recent advances focus on monocular depth estimation, leveraging self-supervised learning algorithms such as Monodepth2 and AF-SfMLearner, which infer depth from video sequences by exploiting temporal consistency. However, the real-world surgical environment introduces various visual disturbances—illumination variability, motion blur, smoke, and sensor noise—that impair model performance. Existing benchmarks primarily evaluate accuracy under ideal conditions, neglecting robustness in adverse scenarios. This gap hampers the deployment of reliable depth estimation systems in clinics. Consequently, there is a pressing need for standardized robustness assessments tailored to endoscopic settings, which this study aims to address by proposing a dedicated benchmark and novel metrics.

Despite significant progress in monocular depth estimation, models often exhibit fragility when exposed to visual corruptions typical in surgery. These disturbances—such as uneven lighting, motion artifacts, and occlusions—cause unpredictable performance drops, risking surgical safety. Current evaluation metrics focus on accuracy on clean data, failing to capture stability under real-world conditions. This discrepancy limits the clinical translation of these models. The core challenge lies in designing a comprehensive, quantifiable measure of robustness that can compare different models objectively and guide their improvement. Without such metrics, it remains difficult to develop models capable of maintaining reliable depth perception amidst the complex visual environment of endoscopic procedures.

This work introduces the Depth Estimation Robustness Score (DERS), a novel composite metric that integrates error, accuracy, and robustness into a single score. Key innovations include: 1) synthesizing a diverse set of corruptions based on real intraoperative disturbances, 2) normalizing traditional error metrics relative to uncorrupted data, 3) weighting accuracy thresholds to reflect clinical importance, and 4) quantifying performance variability across corruption levels to assess stability. The framework systematically evaluates two state-of-the-art models, revealing their vulnerabilities and strengths. This approach surpasses conventional benchmarks by providing a multi-faceted, clinically relevant assessment of model reliability, fostering the development of more resilient depth estimation algorithms suitable for surgical environments.

• �� Data augmentation: Starting from SCARED, introduce 16 corruption types (blur, noise, brightness, etc.) at five severity levels using the imagecorruptions library. • DERS formulation:
• Error component (E): normalize traditional metrics (AbsRel, RMSE) against clean data errors.
• Accuracy component (A): compute weighted average of thresholds (a1, a2, a3) indicating prediction proximity.
• Robustness component (R): calculate standard deviation of metrics across corruption levels, scaled by a tunable factor λ.
• Final score: DERS = E / A × e^(-R), where lower scores indicate better robustness.
• �� Model evaluation: Apply to MonoDepth2 and AF-SfMLearner, using GPU-based testing, analyze performance across corruption types and severity levels, and compare DERS scores to assess stability.

The evaluation uses the extended SCARED-C dataset with 16 corruption types, each at five severity levels. Models are trained on clean data and tested across corruptions, measuring traditional metrics and computing DERS. The experiments include ablation studies to assess the impact of each corruption type, sensitivity analysis across severity levels, and comparison between models. Results demonstrate that MonoDepth2 maintains higher robustness, with lower DERS scores, especially under blur and noise. The experiments validate the effectiveness of DERS in capturing performance stability and guide future model design. All tests are conducted on RTX4090 GPU, ensuring computational consistency.

MonoDepth2 consistently outperforms AF-SfMLearner in robustness, with an average DERS of 5.55 versus 5.66. Corruptions like Gaussian noise and motion blur cause significant performance degradation, with DERS increases of over 2 points. The results highlight that robustness varies substantially across corruption types, emphasizing the need for models trained with diverse disturbances. Notably, certain corruptions like JPEG compression have minimal impact, indicating their lesser relevance in clinical scenarios. The comprehensive evaluation underscores the importance of robustness metrics for real-world deployment, revealing model vulnerabilities and guiding targeted improvements.

The proposed framework can be used by researchers and clinicians to develop and select depth estimation models with higher reliability in surgical environments. It supports the design of algorithms that maintain performance despite visual impairments, crucial for robotic surgery, navigation, and augmented reality-assisted procedures. Long-term, this approach can facilitate the integration of robust depth perception systems into surgical robots, improving precision, safety, and patient outcomes. The methodology also encourages standardization in robustness evaluation across medical imaging tasks, fostering broader clinical adoption of deep learning solutions.

The corruptions are synthetically generated, which may not fully replicate the complexity of real intraoperative disturbances, potentially limiting clinical applicability. Evaluation is restricted to two models, and broader testing with multi-modal or multi-view systems is needed. Corruption parameters are somewhat subjective; future work should incorporate real surgical data for validation. Additionally, computational costs and real-time performance constraints must be considered for clinical deployment. The current framework does not address dynamic or multi-modal disturbances, which are common in actual surgeries, representing an area for future research.