FlexVLN: Flexible Adaptation for Diverse Vision-and-Language Navigation Tasks

VLN evolved from R2R’s sequential instructions toward RxR’s multilingual language, CVDN’s dialogue, and the object-oriented REVERIE and SOON benchmarks. Supervised systems such as BEVBert, GridMM, and ScaleVLN have improved in-domain results but remain dataset-specific. LLM-based agents including NavGPT, DiscussGPT, and MapGPT broaden language coverage, yet often suffer from unstable low-level control, high invocation cost, and insufficient spatial discrimination.

Given instruction I, current position, four-view observation, and history, an agent must traverse graph G={V,E} and locate the requested object. The challenge is a mismatch between high-level, dataset-dependent language and models trained on fine-grained routes. LLMs lack exact traversability and node information, while supervised followers struggle with unseen instruction styles. A useful system must transfer semantics without sacrificing execution reliability.

• �� Hierarchical planning: the LLM specifies a semantic destination and fine-grained guidance rather than selecting every neighboring node.
• �� Feasibility verification: Qwen2-VL rejects impossible commands such as passing through a closed door.
• �� Ensemble execution: BEVBert, GridMM, and ScaleVLN act by consensus; GPT-4o-mini handles conflicts.
• �� Closed-loop feedback: executed trajectories are converted to history, and planning continues until “Finished!” is produced.

• �� Perception: four 90-degree views at 0°, 90°, 180°, and 270° are summarized by InternVL; Faster R-CNN uses simulator depth to detect objects within three meters.
• �� Planning: the LLM receives instruction I, history H, and observation Ot, then outputs guidance using phrases such as “go forward,” “turn left,” and “go into.”
• �� Verification: Qwen2-VL checks the directional visual evidence; infeasible guidance returns to planning with an explanation.
• �� Execution: three followers predict an action at each step. Agreement is executed directly; disagreement is resolved from six textual directional options by GPT-4o-mini.
• �� Completion: the trajectory updates history, the Planner decides whether the destination is reached, and the Object Locator identifies the final object.

The evaluation uses REVERIE, SOON, and CVDN-target as out-of-domain datasets, explicitly without additional target-domain training or fine-tuning. Components include InternVL, Faster R-CNN, Qwen2-VL, BEVBert, GridMM, ScaleVLN, and GPT-4o-mini. The provided manuscript text does not contain the result tables, metric values, ablations, or hardware settings; therefore only the reported qualitative ranking can be verified here.

The authors report that FlexVLN substantially surpasses previous methods on all three out-of-domain benchmarks and approaches models trained directly on the target data. Mechanistically, it avoids ambiguous direct node selection, preserves route execution around obstacles, and reduces LLM calls by planning in segments. Exact SR, RGS, SPL, improvement margins, and ablation values are unavailable in the supplied excerpt and should not be inferred.

A household robot could interpret “go to the first-floor lounge and turn on the lamp,” convert it into familiar navigation guidance, and execute it with an existing follower. Hotel, hospital, and warehouse robots could similarly reuse a base controller across changing instruction styles. Deployment requires multiview perception, a navigable topology, object detection, and access to multimodal language models.

Multiple large models create latency, communication, and API-cost concerns that may limit real-time deployment. Failure cases include hallucinated scene descriptions, hidden obstacles, absent target objects, and repeated replanning. The current evidence comes mainly from discrete VLN simulation, not continuous control, moving pedestrians, physical interaction, or real sensor noise. Future work should add cost-aware routing, confidence calibration, small-model distillation, and physical-robot tests.