Bridging the Indoor-Outdoor Gap: Vision-Centric Instruction-Guided Embodied Navigation for the Last Meters
Proposes BridgeNav, a vision-based, instruction-guided indoor-outdoor navigation system achieving 89.55% success rate without priors.
Key Findings
Methodology
This work introduces a vision-centric navigation framework integrating a latent intention inference module and an optical flow-guided dynamic perception module. Visual features are encoded via Vision Transformer, while a multimodal large language model (Qwen2.5-VL-3B) facilitates cross-modal interaction. The latent intention module dynamically attends to task-relevant regions during navigation phases, and the optical flow module predicts future visual changes by estimating flow fields with RAFT. Training employs a two-stage process: first optimizing intention inference, then navigation and perception. The dataset comprises 55K street-view images, generated via trajectory-conditioned video synthesis, enabling robust training without external priors.
Key Results
- The proposed method achieves a success rate of 89.55% within 0.3 meters, outperforming previous SOTA approaches by approximately 20%. Path deviation is reduced to 9.77%. Ablation studies show that removing intention inference or dynamic perception modules decreases success rate by 15-20%.
- Across diverse unseen urban environments, the model maintains high success rates, demonstrating strong generalization. The integration of visual cues and instruction understanding effectively bridges outdoor-indoor transition gaps.
- Compared to traditional methods relying on precise coordinates or semantic maps, this approach solely depends on egocentric visual observations guided by natural language instructions, significantly enhancing practicality for real-world deployment.
Significance
This research addresses the critical challenge of outdoor-to-indoor navigation without relying on external priors, a long-standing bottleneck in autonomous robotics. By leveraging visual and language cues, it enables robots to operate seamlessly in complex urban settings, supporting applications like last-mile delivery and autonomous assistance. The open-source dataset and video synthesis pipeline also provide valuable resources for future research, fostering advancements in multi-modal perception and navigation. The approach reduces dependency on costly map data, enhances privacy compliance, and paves the way for scalable, adaptable autonomous systems in smart cities.
Technical Contribution
The paper introduces a novel, fully visual instruction-driven navigation framework that integrates latent intention inference and optical flow-based dynamic perception, enabling continuous focus on task-relevant regions. The use of a large multimodal language model for cross-modal understanding and trajectory-conditioned video generation for data augmentation are key innovations. The architecture allows for end-to-end training and real-time decision-making, surpassing existing methods that depend on external location or semantic maps. The combination of these components results in a robust, scalable system capable of precise outdoor-indoor transition without prior maps.
Novelty
This is the first work to formulate a completely prior-free, vision-centric instruction-guided navigation task bridging outdoor and indoor environments. It innovatively combines latent intention attention, optical flow prediction, and trajectory-conditioned video synthesis, creating a new paradigm that relies solely on egocentric visual and linguistic cues. Unlike prior works that depend on GPS, semantic maps, or detailed instructions, this approach demonstrates high success in complex urban scenarios, marking a significant leap forward.
Limitations
- The system's performance degrades under extreme visual occlusion, poor lighting, or adverse weather conditions, as visual cues become unreliable.
- Handling highly cluttered or densely built environments remains challenging due to limited visual resolution and occlusion effects.
- Training requires extensive high-quality street-view data, which is costly to acquire and annotate; scalability to diverse global environments needs further exploration.
Future Work
Future directions include integrating reinforcement learning for adaptive path planning, enhancing robustness against visual disturbances, and extending the dataset to more diverse urban settings. Real-time deployment and multi-agent coordination are also promising avenues, aiming to create fully autonomous, scalable urban robots capable of complex navigation tasks.
AI Executive Summary
The rapid growth of urban robotics demands robust solutions for seamless outdoor-to-indoor navigation, especially in complex city environments where traditional methods relying on GPS or semantic maps often falter. Existing approaches typically depend on external priors, such as precise coordinates or detailed semantic maps, which are difficult to obtain, especially considering privacy constraints and data collection costs. These limitations hinder the deployment of autonomous agents in real-world scenarios like last-mile delivery, where precise entry into buildings is essential.
To address this challenge, the authors propose BridgeNav, a novel vision-centric, instruction-guided navigation framework that operates without any external priors. The core innovation lies in leveraging egocentric visual observations combined with natural language instructions to guide the agent through the entire outdoor-to-indoor transition. The system employs a latent intention inference module that dynamically attends to task-relevant regions, shifting focus from broad scene understanding to precise entrance detection as the agent approaches its target. Complementing this, an optical flow-guided dynamic perception module predicts future visual changes, enabling the agent to 'imagine' upcoming observations and plan accordingly.
The architecture integrates a Vision Transformer for visual encoding, a large multimodal language model (Qwen2.5-VL-3B) for cross-modal understanding, and a decoder for trajectory prediction. To facilitate training, the authors built a comprehensive dataset of 55,000 street-view images, generated via a trajectory-conditioned video synthesis pipeline that simulates realistic urban navigation scenarios. Experimental results demonstrate that BridgeNav achieves a success rate of 89.55% within 0.3 meters, significantly outperforming existing methods. The success of this approach not only advances academic understanding but also opens new avenues for practical deployment of autonomous agents in urban environments.
Despite its strengths, the system faces challenges under extreme visual disturbances and in highly cluttered environments. Future work aims to incorporate reinforcement learning, expand dataset diversity, and optimize real-time performance. Overall, this research marks a significant step toward fully autonomous, scalable urban navigation systems capable of complex outdoor-indoor transitions without relying on external maps or priors, promising transformative impacts on logistics, assistance robotics, and smart city infrastructure.
Deep Dive
Glossary
Vision Transformer (视觉变换器)
一种基于Transformer架构的深度学习模型，用于高效编码图像特征，提升视觉理解能力。
在论文中用于编码视觉观察信息。
潜在意图推断 (Latent Intention Inference)
通过模型动态关注导航过程中最相关区域，指导路径规划和目标识别。
核心模块之一，动态调整关注区域。
光流 (Optical Flow)
描述连续帧之间像素运动的向量场，用于预测未来观察变化。
用于增强模型对未来视觉变化的“想象”。
Qwen2.5-VL-3B
一种多模态大规模预训练模型，支持视觉与语言的跨模态理解。
实现视觉信息与指令的融合。
轨迹条件视频生成
利用已知路径信息合成逼真的城市街景视频，用于增强训练数据。
解决数据不足问题，提升模型泛化。
Open Questions Unanswered questions from this research
- 1 如何进一步提升模型在极端环境（如强光、遮挡）下的鲁棒性？
- 2 是否可以结合强化学习优化路径决策？
- 3 未来如何实现多机器人协作导航？
Applications
Immediate Applications
智能物流配送
无人配送机器人利用该技术在复杂城市环境中自主导航，提升配送效率，减少人力成本。
智能导览系统
导览机器人或无人导游在景区或商场中自主引导访客，无需详细地图或定位信息。
Long-term Vision
智慧城市基础设施
实现城市中自主机器人无缝导航，优化交通管理和公共服务，推动智能城市建设。
Abstract
Embodied navigation holds significant promise for real-world applications such as last-mile delivery. However, most existing approaches are confined to either indoor or outdoor environments and rely heavily on strong assumptions, such as access to precise coordinate systems. While current outdoor methods can guide agents to the vicinity of a target using coarse-grained localization, they fail to enable fine-grained entry through specific building entrances, critically limiting their utility in practical deployment scenarios that require seamless outdoor-to-indoor transitions. To bridge this gap, we introduce a novel task: out-to-in prior-free instruction-driven embodied navigation. This formulation explicitly eliminates reliance on accurate external priors, requiring agents to navigate solely based on egocentric visual observations guided by instructions. To tackle this task, we propose a vision-centric embodied navigation framework that leverages image-based prompts to drive decision-making. Additionally, we present the first open-source dataset for this task, featuring a pipeline that integrates trajectory-conditioned video synthesis into the data generation process. Through extensive experiments, we demonstrate that our proposed method consistently outperforms state-of-the-art baselines across key metrics including success rate and path efficiency.