EndoOmni: Zero-Shot Cross-Dataset Depth Estimation in Endoscopy by Robust Self-Learning from Noisy Labels
EndoOmni employs teacher-student self-learning with confidence-guided robust loss to achieve zero-shot cross-dataset endoscopy depth estimation, reducing absolute relative error by 33%.
Key Findings
Methodology
EndoOmni utilizes a teacher-student framework where the teacher model is trained on large-scale labeled data using scale-and-shift invariant (SSI) loss. It then generates pseudo labels and confidence maps for unlabeled data. The student model learns from both sources, guided by a weighted scale-and-shift invariant (WSSI) loss that incorporates label confidence and pseudo-label consistency. This approach effectively mitigates noise, enhances generalization across diverse datasets, and enables zero-shot cross-domain depth estimation. The core algorithms include SSI loss, confidence estimation formulas (Eqs. 1-10), and dynamic weighting strategies that focus learning on high-confidence regions.
Key Results
- On the Hamlyn dataset, EndoOmni achieves an absolute relative error (AbsRel) of 0.125, outperforming previous methods by 33%. On SERV-CT, it reaches 0.053, surpassing SOTA by 34%. The model demonstrates strong zero-shot generalization to unseen datasets, maintaining high accuracy without fine-tuning.
- Fine-tuning results show that the model retains superior performance in metric depth estimation, with AbsRel of 0.127 on Hamlyn and 0.345 on bronchoscopic data, validating its effectiveness as a foundational model.
- Ablation studies confirm that confidence-guided WSSI loss significantly improves robustness, reducing the impact of noisy pseudo labels and enhancing learning in challenging regions.
Significance
This work addresses critical challenges in medical image depth estimation—data scarcity and noisy labels—by introducing a scalable, robust framework capable of zero-shot generalization across diverse endoscopic datasets. It advances the field by providing a foundation model that can be fine-tuned for high-precision tasks, facilitating applications like navigation, reconstruction, and augmented reality in minimally invasive surgery. The approach paves the way for deploying deep learning models in real-world clinical scenarios, where data variability and noise are inevitable, ultimately contributing to safer and more effective surgical procedures.
Technical Contribution
The paper introduces a novel teacher-student self-learning paradigm combined with confidence-aware robust loss functions, notably the WSSI loss, which adaptively weights pixel-wise errors based on label confidence and pseudo-label consistency. This design effectively suppresses noise influence, improves convergence stability, and enhances cross-dataset generalization. The large-scale multi-source dataset curated for training further distinguishes this work, enabling the model to learn robust features applicable across different endoscopic environments.
Novelty
This is the first study to develop a foundation model for zero-shot cross-dataset depth estimation in endoscopy, integrating confidence-guided robust self-learning with a large, diverse dataset. Unlike prior methods limited to in-domain or synthetic data, EndoOmni achieves significant out-of-domain performance improvements, setting a new benchmark for medical depth estimation. Its innovative loss design and training strategy address longstanding issues of label noise and data scarcity.
Limitations
- Despite robustness, the model still struggles in extreme scenarios with severe occlusion or noise, indicating room for further enhancement in confidence estimation and multi-modal integration.
- Training relies on extensive multi-source datasets, which are costly and time-consuming to compile, potentially limiting scalability.
- Real-time inference and deployment in clinical settings require further optimization to meet latency and resource constraints.
Future Work
Future directions include integrating multi-modal data such as ultrasound or optical coherence tomography to improve accuracy in challenging regions. Developing semi-supervised or unsupervised extensions could reduce dependence on large annotated datasets. Additionally, exploring model compression and real-time inference will facilitate clinical deployment, bringing this technology closer to routine surgical workflows.
AI Executive Summary
Deep learning-based depth estimation in endoscopy has the potential to revolutionize minimally invasive surgery by enabling precise navigation, 3D reconstruction, and augmented reality overlays. However, existing models are limited by the scarcity of high-quality labeled data and the noise inherent in medical annotations. Traditional approaches often focus on in-domain datasets, which fail to generalize well to unseen environments, restricting their clinical utility.
This paper introduces EndoOmni, a pioneering foundation model designed for zero-shot cross-dataset depth estimation in endoscopy. The core innovation lies in a teacher-student self-learning framework that leverages large-scale, diverse datasets, including unlabeled data, to learn robust features. The teacher model, trained with a scale-and-shift invariant loss, generates pseudo labels and confidence maps, which guide the student model's training. To address the noisy labels typical in medical data, the authors propose a weighted loss function—WSSI—that dynamically emphasizes high-confidence regions and pseudo-label consistency, effectively suppressing noise interference.
Extensive experiments demonstrate that EndoOmni achieves remarkable performance, reducing absolute relative error by 33% on the Hamlyn dataset and 34% on SERV-CT, outperforming state-of-the-art methods. The model maintains strong generalization capabilities, performing well on unseen datasets without fine-tuning. When fine-tuned for metric depth estimation, it continues to outperform existing approaches, confirming its role as a robust foundation model.
This work significantly advances the field by providing a scalable, noise-robust, cross-domain depth estimation framework tailored for medical imaging. Its implications extend to improving surgical navigation, enhancing 3D visualization, and enabling new AR-assisted procedures. Future research will focus on integrating multi-modal data, reducing annotation costs, and optimizing for real-time deployment, aiming to bring intelligent endoscopic systems into routine clinical practice.
Deep Dive
Abstract
Single-image depth estimation is essential for endoscopy tasks such as localization, reconstruction, and augmented reality. Most existing methods in surgical scenes focus on in-domain depth estimation, limiting their real-world applicability. This constraint stems from the scarcity and inferior labeling quality of medical data for training. In this work, we present EndoOmni, the first foundation model for zero-shot cross-domain depth estimation for endoscopy. To harness the potential of diverse training data, we refine the advanced self-learning paradigm that employs a teacher model to generate pseudo-labels, guiding a student model trained on large-scale labeled and unlabeled data. To address training disturbance caused by inherent noise in depth labels, we propose a robust training framework that leverages both depth labels and estimated confidence from the teacher model to jointly guide the student model training. Moreover, we propose a weighted scale-and-shift invariant loss to adaptively adjust learning weights based on label confidence, thus imposing learning bias towards cleaner label pixels while reducing the influence of highly noisy pixels. Experiments on zero-shot relative depth estimation show that our EndoOmni improves state-of-the-art methods in medical imaging for 33\% and existing foundation models for 34\% in terms of absolute relative error on specific datasets. Furthermore, our model provides strong initialization for fine-tuning metric depth estimation, maintaining superior performance in both in-domain and out-of-domain scenarios. The source code is publicly available at https://github.com/TianCuteQY/EndoOmni.