Vision Foundation Models with Synthetic-Only Training for Monocular Spacecraft Pose Estimation
Using DINOv3 for monocular spacecraft pose estimation, achieving lowest rotation errors.
Key Findings
Methodology
The study employs a heatmap-based pose estimation architecture, using a large self-supervised ViT model (DINOv3) instead of smaller convolutional and ViT encoders. Model parameters expanded from 300M to 840M with no saturation observed.
Key Results
- On the SPEED+ dataset, using DINOv3 840M model, achieved mean rotation errors of 1.17° on lightbox and 1.56° on sunlamp, outperforming EagerNet's 2.66° and 1.75°.
- On Jetson Orin NX 16GB, single inference takes 133.8 ms with 32.0 W power draw, demonstrating feasibility of embedded inference.
- DINOv3 840M model, adapted with LoRA, outperforms previous models across lightbox and sunlamp domains, trained only on synthetic data.
Significance
This research makes significant advances in spacecraft pose estimation, achieving the lowest rotation errors using only synthetic data. It provides a more accurate estimation method for future space missions, reducing reliance on real data.
Technical Contribution
By using the large self-supervised ViT model DINOv3, the study significantly improves pose estimation accuracy while keeping other architectural elements constant. Introduced LoRA for fine-tuning large models, demonstrating feasibility on embedded hardware.
Novelty
This is the first use of DINOv3 in spacecraft pose estimation, significantly reducing rotation errors through LoRA fine-tuning, showcasing the potential of large models in this field.
Limitations
- The model was trained and tested on a single target satellite, which may not generalize to other targets.
- Strong assumptions on target geometry and keypoints limit the model's generality.
Future Work
Future research could explore applications in multi-target environments and use larger foundation models for more complex missions.
AI Executive Summary
This study proposes an improved spacecraft pose estimation method using the large self-supervised ViT model DINOv3, replacing traditional smaller convolutional and ViT encoders. Trained solely on synthetic data, the study achieves the lowest rotation errors on the SPEED+ dataset, with 1.17° on lightbox and 1.56° on sunlamp, outperforming existing EagerNet models.
The research demonstrates the feasibility of embedded inference on Jetson Orin NX 16GB hardware, with single inference taking 133.8 ms and 32.0 W power draw. This provides a more accurate pose estimation method for future space missions, reducing reliance on real data.
While significant progress was made on a specific target satellite, the model's generality needs further validation in multi-target environments. Future research could explore applications in more complex missions and the potential of using larger foundation models.
Deep Analysis
Background
Spacecraft pose estimation is a critical technology in space missions, traditionally relying on real data and small models. Recent developments in synthetic datasets and large models have prompted researchers to explore more efficient methods.
Core Problem
Current pose estimation methods perform poorly under non-cooperative targets and complex lighting conditions, increasing mission costs and risks due to reliance on real data.
Innovation
The study introduces the large self-supervised ViT model DINOv3, combined with LoRA for fine-tuning, significantly improving pose estimation accuracy and reducing reliance on real data.
Methodology
- �� Use DINOv3 model to replace traditional encoders
- �� Apply LoRA for fine-tuning
- �� Train and test on SPEED+ dataset
- �� Conduct embedded inference tests on Jetson Orin NX 16GB
Experiments
Experiments were conducted on the SPEED+ dataset, evaluating model performance using lightbox and sunlamp test sets. Experimental settings included model parameter expansion and different inference hardware tests.
Results
The DINOv3 840M model achieved rotation errors of 1.17° and 1.56° on lightbox and sunlamp test sets, respectively, significantly outperforming existing models.
Applications
The method can be used in future space missions, particularly in non-cooperative target missions requiring high-precision pose estimation.
Limitations & Outlook
The model's generality in multi-target environments remains unverified, and future research should explore applications in more complex missions.
Plain Language Accessible to non-experts
Imagine playing a drone game where precise control is crucial. Traditional methods are like using an old controller, not very precise. This study is like upgrading to the latest controller, more responsive and accurate. By using synthetic data and advanced algorithms, this new controller not only improves your game performance but also adapts to various complex game environments.
ELI14 Explained like you're 14
Imagine playing a super cool drone game. You need to control the drone in various complex environments, and this study is like giving you a super smart drone control system. It can accurately estimate the drone's pose under various lighting conditions, making you a pro in the game! Isn't that awesome?
Glossary
DINOv3 (Self-supervised Vision Transformer)
A large self-supervised learning model used for vision tasks.
Used to replace traditional smaller convolutional and ViT encoders.
LoRA (Low-rank Adapter)
A fine-tuning technique allowing adjustment of a small portion of parameters while freezing most of the model.
Used for fine-tuning the DINOv3 model.
SPEED+ (Satellite Pose Estimation Dataset)
A synthetic dataset for spacecraft pose estimation, including lightbox and sunlamp test sets.
Used to evaluate model performance.
Jetson Orin NX
An embedded hardware platform used for testing model inference performance.
Used to evaluate model feasibility in embedded environments.
Pose Estimation
Determining the position and orientation of a spacecraft in space.
The core task of the study.
Open Questions Unanswered questions from this research
- 1 How to apply the model in multi-target environments?
- 2 How does the model perform in complex missions?
- 3 How to further reduce the computational cost of the model?
Applications
Immediate Applications
Non-cooperative Target Missions
Provides high-precision pose estimation without relying on real data.
Long-term Vision
Automated Space Missions
Achieve unmanned space mission operations through more accurate pose estimation.
Abstract
We present an improvement on previous spacecraft pose estimation architectures that results in the lowest published mean rotation errors we know of on the SPEED+ lightbox and sunlamp test sets for a known, non-cooperative spacecraft. By using a previously established heatmap-based pose estimation architecture and adapting a large self-supervised ViT foundation model (DINOv3) in place of the smaller convolutional and ViT encoders of previous work, we show that pose estimation accuracy improves from 300M to 840M parameters with no saturation yet observed. We also evaluate our 840M model on a Jetson Orin NX 16GB, measuring single-pass network inference at 133.8 ms per crop with a board draw of 32.0 W. These measurements demonstrate embedded inference feasibility on a processor family with orbital flight heritage. Our resulting model outperforms previous models across lightbox and sunlamp domains while training only on synthetic data. Our best model, using DINOv3 840M adapted with LoRA as the encoder (rank 64, three-seed ensemble with four-rotation test-time augmentation), results in $1.56^\circ$ mean rotation error on sunlamp and $1.17^\circ$ on lightbox, compared to the previous best mean rotation errors we know of on these test sets, $2.66^\circ$ and $1.75^\circ$ by EagerNet.