GeoDistill-Refine: Silhouette-First Geometry Distillation for Annotation-Free Spacecraft Segmentation
GeoDistill-Refine employs multi-prompt fusion and signed distance fields, boosting annotation-free spacecraft segmentation IoU by 4.56% and Boundary F1 by 13.80%.
Key Findings
Methodology
This approach constructs pseudo-masks offline using the SAM 3 model with six fixed prompts, then fuses them via an unweighted 50% majority vote to generate stable teacher labels. The lightweight TinyUNet student first learns the silhouette (stage I), then incorporates geometric supervision through signed distance fields (SDF), skeletons, and area constraints (stage II). To mitigate unreliable pseudo-geometry, a sample-level gate based on prompt agreement, valid prompt ratio, and area plausibility dynamically adjusts the influence of geometric losses. The training process involves two phases: initial silhouette learning, followed by geometric refinement, resulting in significant performance gains on the SpaceSense-Bench dataset, with IoU increasing by 4.56% and Boundary F1 by 13.80%.
Key Results
- On the SpaceSense-Bench HJM lockbox set, GeoDistill-Refine outperforms a plain pseudo-label student, with a 0.0456 increase in Image IoU and a 0.1380 increase in Boundary F1, demonstrating improved generalization to unseen spacecraft.
- External evaluations on SPEED+ Lightbox, Sunlamp, and TANGO datasets show that the model maintains high regional overlap and achieves notable gains in boundary accuracy and foreground precision, indicating robustness across diverse imaging conditions.
- The multi-prompt fusion strategy reduces pseudo-label uncertainty, while the integration of geometric supervision via SDF and skeletons effectively suppresses geometric errors, leading to sharper boundaries and better structural details.
Significance
This work addresses the critical challenge of achieving high-precision spacecraft segmentation without manual annotations, a necessity for space missions where labeling is costly and impractical. By leveraging foundation models and a novel geometric distillation framework, it enables robust perception in unstructured and unseen environments. The approach significantly reduces reliance on labeled data, accelerates deployment, and enhances the autonomy of space systems, paving the way for more scalable and adaptable space exploration and debris management solutions.
Technical Contribution
The paper introduces a multi-prompt pseudo-mask fusion mechanism that stabilizes pseudo-labels, combined with the integration of signed distance fields, skeletons, and area constraints for geometric refinement. The sample-level gate dynamically filters unreliable geometric supervision, ensuring robustness. The two-stage training strategy—initial silhouette learning followed by geometric refinement—improves stability and accuracy. The lightweight TinyUNet architecture, with only 0.263 million parameters, achieves real-time inference (~1.1 ms per image) on high-end GPUs, making it suitable for edge deployment. These innovations collectively advance the state-of-the-art in annotation-free spacecraft segmentation.
Novelty
This study is the first to combine multi-prompt fusion with signed distance fields and skeleton-based geometric supervision in an annotation-free setting for spacecraft segmentation. The introduction of a sample-level reliability gate to filter pseudo-geometric errors is novel, effectively addressing the propagation of teacher inaccuracies. Unlike prior works that rely solely on confidence filtering or augmentation consistency, this approach explicitly models geometric uncertainty and selectively incorporates reliable geometric cues, significantly improving segmentation quality in challenging scenarios.
Limitations
- The method heavily depends on the quality of SAM 3 pseudo-masks; in scenarios with poor prompt design or complex backgrounds, the pseudo-labels may still be unreliable, limiting performance.
- Adding geometric supervision increases training complexity and computational load, which may hinder scalability or real-time deployment in resource-constrained environments.
- The current model is optimized for single-frame RGB inputs and binary segmentation; extending to multi-class or multi-frame scenarios requires further development.
- Performance in extremely cluttered, occluded, or low-visibility conditions remains to be validated, and the method may need adaptation for such challenging environments.
Future Work
Future directions include integrating multi-modal data (depth, IR) to improve robustness, exploring self-supervised strategies to reduce reliance on pseudo-labels, and extending the framework to multi-class segmentation tasks. Additionally, optimizing the geometric supervision components for faster training and inference, as well as deploying the model on embedded systems for real-time space applications, are promising avenues. Further research into handling occlusion, clutter, and low-visibility scenarios will enhance the method's applicability in complex space environments.
AI Executive Summary
Spacecraft segmentation is a foundational task for autonomous space operations, including rendezvous, debris removal, and on-orbit servicing. Traditional approaches rely heavily on manual annotations, which are costly and impractical given the scarcity of labeled space imagery. Recent advances in foundation models like SAM have opened new avenues for annotation-free segmentation, but their predictions are often unstable and contain geometric inaccuracies, especially when applied to unseen targets.
This paper introduces GeoDistill-Refine, a novel framework that leverages multiple prompts to generate stable pseudo-labels for spacecraft segmentation without manual annotations. The core idea involves querying a frozen SAM 3 model with six fixed prompts, then fusing the resulting masks via an unweighted 50% majority vote to produce a reliable teacher pseudo-mask. The lightweight TinyUNet model is trained in two stages: first to learn the silhouette, then to incorporate geometric details through signed distance fields (SDF), skeletons, and area constraints. To address the unreliability of pseudo-geometry, the authors design a sample-level gate based on prompt agreement, valid prompt ratio, and area plausibility, which dynamically filters the geometric supervision.
The experimental results demonstrate the effectiveness of this approach. On the SpaceSense-Bench HJM lockbox set, GeoDistill-Refine outperforms a baseline pseudo-label student, with an increase of 0.0456 in Image IoU and 0.1380 in Boundary F1, indicating better generalization to unseen spacecraft. External evaluations on datasets with challenging illumination conditions, such as SPEED+ and Sunlamp, as well as the TANGO dataset, confirm the robustness and adaptability of the method across different environments.
The key innovations include the multi-prompt fusion strategy, which stabilizes pseudo-labels; the integration of geometric supervision via signed distance fields and skeletons; and the sample-level reliability gate, which filters unreliable geometric cues. These contributions collectively enable high-precision, annotation-free spacecraft segmentation suitable for deployment on edge devices, with a parameter count of only 0.263 million and inference time around 1.1 ms per image.
Overall, this work advances the state-of-the-art in unsupervised space perception, reducing reliance on manual labels and enhancing robustness in unseen and challenging scenarios. It opens new pathways for autonomous space missions, where rapid, accurate perception is crucial. Future research will likely focus on multi-modal data integration, multi-class segmentation, and real-time deployment in complex space environments, further broadening the impact of this promising approach.
Deep Analysis
Background
空间目标分割在空间机器人、目标追踪和空间垃圾管理中具有重要意义。传统方法依赖大量手工标注，成本高且难以扩展。近年来，深度学习模型如U-Net、Mask R-CNN在地面场景中取得显著进展，但在空间环境中，背景复杂、目标尺度变化大、低对比度等问题依然存在。基础模型如SAM（Segment Anything Model）问世，为无标注学习提供了新思路。SAM通过大规模预训练，能在不同提示下生成目标掩码，为空间目标提供伪标签。然而，伪标签的稳定性和几何精度不足，尤其在未见目标和复杂背景中表现不佳。现有研究多关注伪标签融合与蒸馏，但缺乏有效的几何误差抑制机制。本论文在此基础上，提出结合多提示融合和几何距离场的无标注空间目标分割框架，旨在提升模型鲁棒性和泛化能力，为空间自主感知提供技术支撑。
Core Problem
核心问题在于如何在没有手工标注的情况下，利用基础模型生成稳定且几何准确的空间目标掩码。伪标签易受提示设计、背景干扰和模型不确定性影响，导致边界模糊、细长结构丢失和连接错误。这些问题限制了模型在未见目标和复杂场景中的应用效果。传统伪标签蒸馏方法依赖置信度或增强一致性，难以过滤伪几何误差，导致性能受限。如何设计一种机制，既能利用多提示生成的多样伪标签，又能抑制错误传播，成为亟待解决的难题。
Innovation
本研究的创新点包括：1）多提示融合机制：通过六个不同提示的SAM伪掩码未加权投票，增强伪标签稳定性，减少提示敏感性。2）引入签名距离场（SDF）和骨架信息，用于几何细化，提升边界还原能力。3）样本级门控机制：结合提示一致性、有效提示比例和面积合理性，动态调节几何目标的影响，过滤伪几何误差。4）两阶段训练策略：先学习轮廓（Stage I），再引入几何距离场（Stage II），逐步提升模型性能。该方法在保证模型轻量化的同时，大幅改善空间目标的分割效果，特别是在未见目标上的泛化能力。
Methodology
- �� 伪掩码生成：利用离线的SAM 3模型，针对每个空间图像，输入六个固定提示，获得六个候选掩码及其得分。只有得分超过阈值的掩码参与融合。• 多提示融合：通过未加权的50%多数投票，形成教师伪掩码，确保在提示不一致时仍能获得较稳定的目标轮廓。• 轮廓蒸馏：训练TinyUNet的掩码分支，首先学习目标轮廓（Stage I），以减少伪几何误差的影响。• 几何细化：在第一阶段基础上，利用伪距离场、骨架和面积目标，逐步引入几何约束（Stage II），通过样本级门控机制调节几何目标的损失权重，过滤掉不可靠的几何信息。• 样本级门控：结合提示一致性、有效提示比例和面积合理性，计算门控系数，动态调节几何目标的损失影响。• 损失函数：包括二值交叉熵（BBCE）、Dice系数、签名距离场误差（SDF）、骨架误差和面积误差，整体优化模型性能。• 训练流程：先进行轮廓学习（Stage I），再在前一阶段基础上引入几何细化（Stage II），最终模型在空间感知基准上实现优异性能。
Experiments
- �� 数据集：在SpaceSense-Bench、SPEED+和TANGO上进行评估。SpaceSense-Bench包含270张训练图像和60张验证图像，测试集由未见空间目标组成。SPEED+和TANGO提供不同照明条件和图像分布的外部验证。• 训练细节：采用AdamW优化器，Batch size为4，Stage I训练20轮，学习率5×10^-4，Stage II训练10轮，学习率10^-4。伪掩码由SAM 3预先生成，采用未加权多数投票融合。• 评价指标：主要用Image IoU和Boundary F1衡量区域重叠和边界精度，辅以前景精度和召回率。• Ablation研究：比较不同提示融合策略、几何目标引入时机和门控机制效果，验证模型的鲁棒性和泛化能力。
Results
- �� 在HJM锁箱集上，GeoDistill-Refine相较于纯伪标签模型，Image IoU提升0.0456，Boundary F1提升13.80%，验证其在未见空间目标上的优越性。• 在SPEED+ Lightbox和Sunlamp域，模型在区域重叠率和边界细节方面表现优异，显著优于基线模型。• 通过多提示融合策略，伪标签的稳定性大幅提高，模型在复杂背景和细长结构场景中的表现得到改善。• 引入几何距离场和骨架目标，有效抑制伪几何误差的传播，提升边界细节还原能力。
Applications
- �� 该方法适用于空间机器人自主感知、目标追踪、空间垃圾清理等任务，尤其在缺乏手工标注的场景中表现出色。• 只需单帧RGB图像和基础模型伪标签，即可实现高精度空间目标分割，便于边缘设备部署。• 未来可结合多模态信息（如深度、红外）扩展应用范围，提升在复杂环境中的鲁棒性。
Limitations & Outlook
- �� 依赖SAM 3模型的伪掩码生成，受限于其性能和提示设计，复杂背景或低对比度场景下效果可能下降。• 添加几何距离场和骨架目标增加训练复杂度和计算成本，模型在极端环境下的鲁棒性仍需验证。• 当前模型参数较少，推理速度快，但在极端复杂场景或高分辨率图像中，性能可能受限，未来需优化模型结构。
Plain Language Accessible to non-experts
想象你在一个工厂里，有很多不同的机器和零件。工厂的管理者希望知道每个机器的轮廓和结构，但不想用手工画图。于是，他们用一台特别聪明的机器人（类似SAM模型）帮忙，这台机器人可以根据不同的提示告诉你哪些部分是机器。你给它六个不同的提示，它会画出六个版本的轮廓，然后用投票的方式决定哪个最靠谱。接着，你用这些答案训练一个小机器人（TinyUNet），让它学会识别机器的轮廓。刚开始，小机器人只学会了大概的轮廓，但为了让它更懂得细节，你还用一种特殊的测量工具（签名距离场）和骨架图告诉它哪些部分更重要、更细长。为了避免小机器人被错误的细节误导，你还设计了一个“可靠性门”，根据每次判断的可信度调整学习力度。经过反复训练，小机器人终于可以在没有老师帮忙的情况下，准确找到空间中的目标，无论它们长得多奇怪或背景多复杂。这就像你教一个新朋友认出学校里的所有老师和同学一样，既省事又快，而且还能应对各种新情况！
ELI14 Explained like you're 14
想象你在学校的科学实验室，有一台超级聪明的机器人，它能帮你画出各种奇怪的太空飞船轮廓。可是，这台机器人需要你给它一些提示，比如“这是飞船”或者“带翅膀的空间站”。你给它六个不同的提示，它会用这些提示画出六个不同版本的轮廓，然后你们一起投票，决定哪个最靠谱。接着，你用这个轮廓教一个小机器人，让它学会认出飞船的轮廓。刚开始，小机器人只会画出大概的轮廓，但你还用一种特殊的“距离测量工具”和“骨架图”告诉它哪些部分更细长、更重要。为了避免小机器人被错误的细节迷惑，你还设计了一个“可靠性门”，根据每次判断的可信度调整它的学习力度。经过反复训练，小机器人终于可以在没有老师帮忙的情况下，自己找到空间中的飞船，不管它们长得多奇怪或背景多复杂。这就像你教一个新朋友认出学校里的所有老师和同学一样，既省事又快，而且还能应对各种新情况！
Abstract
Foundation segmentation models can provide supervision for spacecraft imagery without manual training masks, but their predictions vary with textual prompts and may contain geometric errors that are amplified during distillation. This paper presents GeoDistill-Refine, a two-stage framework that transfers offline SAM 3 pseudo-masks to a compact segmentation network. Six fixed prompts are fused by an unweighted 50% vote to stabilize the teacher output. The student first learns the foreground silhouette and is then refined with signed-distance-field, skeleton, and area objectives derived from the pseudo-mask. A sample-level gate, computed from prompt agreement, the valid-prompt ratio, and pseudo-mask area plausibility, reduces the influence of unreliable pseudo-geometry. On the SpaceSense-Bench HJM lockbox set, GeoDistill-Refine improves Image IoU and Boundary F1 by 0.0456 and 0.1380, respectively, over a plain pseudo-label student. External evaluations on the SPEED+ Lightbox and Sunlamp domains and on TANGO show competitive regional overlap together with gains in boundary quality or foreground precision. The deployed TinyUNet contains 0.263 M parameters and requires approximately 1.1 ms per image on an RTX 4090; SAM 3 pseudo-mask construction and the auxiliary geometry branches are used only during training.
References (20)
A Spacecraft Dataset for Detection, Segmentation and Parts Recognition
D. Hoang, Bo Chen, Tat-Jun Chin
Dataset generation and validation for spacecraft pose estimation via monocular images processing
M. Bechini, M. Lavagna, P. Lunghi
Gated-SCNN: Gated Shape CNNs for Semantic Segmentation
Towaki Takikawa, David Acuna, V. Jampani et al.
A Survey on Deep Learning-Based Monocular Spacecraft Pose Estimation: Current State, Limitations and Prospects
Leo Pauly, Wassim Rharbaoui, C. Shneider et al.
GABI: Geometry-Aware Boundary Integration for Spacecraft Segmentation
I. G. Velentzas, Dhruv Ahuja, Panagiotis Tsiotras
Satellite Pose Estimation Challenge: Dataset, Competition Design, and Results
Mate Kisantal, Sumant Sharma, T. Park et al.
SPEED+: Next-Generation Dataset for Spacecraft Pose Estimation across Domain Gap
T. Park, Marcus Märtens, Gurvan Lécuyer et al.
Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection
Shilong Liu, Zhaoyang Zeng, Tianhe Ren et al.
Automatic Segmentation Annotation of Space Target Using Segment Anything Model and Object Detection Prompts
Zhihao Zhang, Zhaohui Dang
Segmentation-based Detection for Efficient Multi-Task Spacecraft Perception
Sivaperuman Muniyasamy, Surendar Devasundaram
Boundary loss for highly unbalanced segmentation
H. Kervadec, Jihene Bouchtiba, Christian Desrosiers et al.
Semi-Supervised Semantic Segmentation Using Unreliable Pseudo-Labels
Yuchao Wang, Haochen Wang, Yujun Shen et al.
SAM 3: Segment Anything with Concepts
Nicolas Carion, Laura Gustafson, Yuan-Ting Hu et al.
Semi-Supervised Semantic Segmentation with Cross Pseudo Supervision
Xiaokang Chen, Yuhui Yuan, Gang Zeng et al.
Pseudo-Label : The Simple and Efficient Semi-Supervised Learning Method for Deep Neural Networks
Dong-Hyun Lee
Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
Antti Tarvainen, Harri Valpola
U-Net: Convolutional Networks for Biomedical Image Segmentation
O. Ronneberger, P. Fischer, T. Brox
Searching for MobileNetV3
Andrew G. Howard, M. Sandler, Grace Chu et al.
Post-Launch Capability Expansion of Vision-Language Models via Prompting for On-Orbit Spacecraft Inspection
Nicholas A. Welsh, Lennon Shikhman, Monty Nehru Attazs et al.
Distance transform regression for spatially-aware deep semantic segmentation
N. Audebert, Alexandre Boulch, B. L. Saux et al.