Self-Imitated Diffusion Policy for Efficient and Robust Visual Navigation
Proposes SIDP, integrating reward-guided self-imitation with diffusion models, achieving 2.5× faster inference and improved robustness in visual navigation.
Key Findings
Methodology
The Self-Imitated Diffusion Policy (SIDP) combines diffusion-based trajectory generation with a reward-guided self-imitation mechanism. During training, the model samples trajectories from its current policy, evaluates their quality via a reward function, and re-weights them using importance sampling. This process aligns the policy distribution with high-reward trajectories, avoiding complex backpropagation through denoising steps. The approach employs importance sampling with KL divergence constraints to optimize the policy, ensuring stable training. During inference, the concentrated distribution allows deterministic sampling via DDIM, removing the need for external trajectory filters, thus significantly accelerating path planning. Additional strategies include goal-agnostic exploration to enhance generalization and a reward-driven curriculum to stabilize training, especially in early stages.
Key Results
- On the InternVLA-N1 benchmark, SIDP outperforms NavDP with an average success rate (SR) increase of approximately 10%, achieving 94.36% in complex scenes. It also surpasses baseline SPL metrics, indicating higher efficiency. On edge hardware (Jetson Orin Nano), inference time drops from 273ms to 110ms, a 2.5× speedup, with no performance loss. Ablation studies confirm the importance of reward-guided self-imitation and goal-agnostic exploration in improving robustness and convergence. The model maintains high success rates under visual perturbations and environment shifts, with less than 5% performance degradation.
- The experimental results demonstrate that SIDP effectively concentrates trajectory distributions, reducing sampling steps and eliminating external filtering. Its robustness tests show resilience against sensory noise and environmental complexity, validating its suitability for real-world robotic deployment. The speedup and stability in low-step sampling highlight its potential for real-time applications in autonomous navigation.
- Overall, SIDP sets new state-of-the-art performance in simulation and real-world tests, offering a practical, efficient, and robust solution for visual path planning. Its ability to operate at high speed on edge devices without sacrificing accuracy marks a significant advancement in autonomous robotics, paving the way for scalable deployment in diverse scenarios.
Significance
This work advances the field of robotic visual navigation by addressing key bottlenecks in diffusion-based path planning. Traditional methods rely heavily on expert demonstrations and external trajectory filtering, which limit scalability and real-time performance. SIDP's innovative use of reward-guided self-imitation internalizes trajectory selection, streamlining the pipeline and enabling end-to-end training and inference. The approach enhances robustness against environmental variability and sensory noise, crucial for real-world deployment. Its high inference efficiency on edge hardware demonstrates practical viability, bridging the gap between academic research and industrial application. By fundamentally improving the stability and speed of diffusion-based planning, SIDP opens new avenues for autonomous systems operating in unstructured, dynamic environments, with broad implications for service robots, autonomous vehicles, and drone navigation.
Technical Contribution
The key technical contribution lies in integrating reward-guided self-imitation into the diffusion modeling framework, replacing external trajectory filtering with an internal distribution concentration mechanism. The method employs importance sampling and KL divergence constraints to optimize the policy, avoiding backpropagation through denoising steps common in reinforcement learning. This results in a stable, efficient training process with explicit error correction capabilities. The model achieves high trajectory diversity during training via goal-agnostic exploration, preventing mode collapse. During inference, the concentrated distribution allows deterministic sampling with fewer denoising steps, significantly reducing latency. These innovations collectively enable a robust, real-time path planning system that surpasses existing diffusion and imitation learning methods.
Novelty
This research is the first to incorporate reward-guided self-imitation within a diffusion model for visual navigation, effectively internalizing trajectory selection and eliminating the need for external filters. Unlike prior work that relies on expert demonstrations or reinforcement learning with complex backpropagation, SIDP leverages importance sampling and distribution concentration to achieve stable, fast inference. Its unique combination of goal-agnostic exploration, reward-driven curriculum, and deterministic sampling distinguishes it from existing diffusion-based planners, setting a new paradigm for efficient, robust, end-to-end path planning in robotics.
Limitations
- The reward function design heavily influences training stability and path quality; poorly designed rewards may lead to suboptimal behaviors.
- Performance may degrade in highly dynamic or unpredictable environments where the static reward model cannot capture all variations.
- Training requires substantial sampling and computational resources, which might challenge deployment on extremely resource-constrained platforms.
Future Work
Future research will focus on integrating multi-modal sensory data, such as LiDAR and semantic maps, to improve adaptability in dynamic environments. Developing adaptive reward functions and online learning mechanisms can further enhance robustness. Extending the framework to multi-agent scenarios and real-time adaptive planning, as well as reducing computational costs, will be key directions. Additionally, exploring transfer learning across different environments could facilitate broader industrial applications, pushing autonomous navigation closer to real-world deployment.
AI Executive Summary
Deep Dive
Plain Language Accessible to non-experts
Imagine you’re trying to find the fastest way out of a maze. Traditional methods are like following a detailed map made by someone else; if the maze changes, you might get lost. Now, think of a cleverer way: you start exploring on your own, trying different paths, and remembering which routes lead to success. Each time you find a good way, you reinforce that choice, and over time, you get better at choosing the best routes without needing a map. This way, you become faster and more adaptable, even if the maze changes. Robots using this method do the same—they learn from their own experience, improving their navigation step by step, making them smarter and quicker at reaching their goals in unpredictable environments.
ELI14 Explained like you're 14
Imagine you’re playing a game where you have to find your way out of a tricky maze. Instead of following a map made by someone else, you try different paths yourself. Each time you find a good route, you remember it and try to do more of that. If you take a wrong turn, you learn from it and avoid that path next time. Over time, you get better at choosing the right way, even if the maze changes a little. Robots do something similar—they try out different paths, learn from their successes, and get faster and smarter at reaching their destination. This way, they don’t need a perfect map or someone telling them what to do—they figure it out on their own, just like you do in a maze!
Abstract
Diffusion policies (DP) have demonstrated significant potential in visual navigation by capturing diverse multi-modal trajectory distributions. However, standard imitation learning (IL), which most DP methods rely on for training, often inherits sub-optimality and redundancy from expert demonstrations, thereby necessitating a computationally intensive "generate-then-filter" pipeline that relies on auxiliary selectors during inference. To address these challenges, we propose Self-Imitated Diffusion Policy (SIDP), a novel framework that learns improved planning by selectively imitating a set of trajectories sampled from itself. Specifically, SIDP introduces a reward-guided self-imitation mechanism that encourages the policy to consistently produce high-quality trajectories efficiently, rather than outputs of inconsistent quality, thereby reducing reliance on extensive sampling and post-filtering. During training, we employ a reward-driven curriculum learning paradigm to mitigate inefficient data utility, and goal-agnostic exploration for trajectory augmentation to improve planning robustness. Extensive evaluations on a comprehensive simulation benchmark show that SIDP significantly outperforms previous methods, with real-world experiments confirming its effectiveness across multiple robotic platforms. On Jetson Orin Nano, SIDP delivers a 2.5$\times$ faster inference than the baseline NavDP, i.e., 110ms VS 273ms, enabling efficient real-time deployment.