GraphWorld: Long-Horizon Planning with World Models for End-to-End Autonomous Driving
GraphWorld leverages latent world models for end-to-end autonomous driving, reducing collision rates by 19.5% on nuScenes 6s horizon.
Key Findings
Methodology
GraphWorld integrates a dynamically constructed Ego-Centric Interaction Graph (ECIG) with a latent world state (W) model. ECIG selectively models neighboring agents via spatial proximity, using cross-attention to propagate relational context. The world state encoder employs GRU to fuse historical motion and map features, while Flow-Matching captures continuous environment evolution, avoiding error accumulation typical of autoregressive models. Multi-modal trajectory queries are conditioned on this latent representation, with importance reweighting enhancing prediction reliability. The training involves multi-stage supervision, including temporal consistency constraints, to promote stable long-term reasoning. The architecture enables efficient, interaction-aware long-horizon planning within an end-to-end framework.
Key Results
- On nuScenes, GraphWorld reduces collision rate by 19.5% over World4Drive at 6 seconds horizon, and achieves a 13.3% speed increase on NAVSIMv1, demonstrating superior long-range planning performance. In Bench2Drive, it significantly lowers collision incidents in multi-agent scenarios. Ablation studies confirm ECIG's neighbor filtering and Flow-Matching's continuous dynamics as critical for performance gains.
- The model maintains high prediction accuracy with lower error margins compared to baselines, especially in complex, dense traffic scenes. Its ability to generalize across datasets underscores its robustness. The approach also exhibits computational efficiency suitable for real-time deployment.
- Further experiments highlight the importance of the latent world state in long-term safety and interaction modeling, with the framework outperforming prior methods in both safety metrics and trajectory diversity.
Significance
This work addresses the core challenge of long-horizon planning in autonomous driving, bridging the gap between perception and decision-making through a unified latent environment understanding. By explicitly modeling interaction dynamics and environment evolution, GraphWorld enhances the safety, robustness, and scalability of end-to-end systems. Its ability to anticipate future states and reason over extended horizons marks a significant step forward, enabling autonomous vehicles to operate reliably in complex, interactive traffic scenarios. This approach paves the way for safer, more intelligent autonomous driving solutions, with broad implications for industry deployment and future research.
Technical Contribution
The paper introduces a novel combination of dynamic ego-centric interaction graphs with a flow-based continuous environment modeling framework. This integration allows for scalable, real-time long-term reasoning without the high computational costs of pixel-based scene generation. The use of Flow-Matching for environment evolution, coupled with importance reweighting of motion hypotheses conditioned on latent states, provides a robust mechanism for long-horizon trajectory planning. The multi-stage training with explicit temporal supervision further ensures the stability and consistency of the learned representations, setting a new standard for end-to-end autonomous driving architectures.
Novelty
This is the first work to combine a dynamically constructed ego-centric interaction graph with a flow-based latent environment model for long-horizon planning in autonomous driving. Unlike prior pixel-level scene generation or short-term prediction models, GraphWorld emphasizes relational, agent-centric representations that encode future interaction dynamics. Its continuous environment evolution modeling via Flow-Matching is a key innovation, enabling stable, efficient long-range reasoning. This approach significantly advances the state-of-the-art in integrating environment understanding with end-to-end decision-making.
Limitations
- Despite its strengths, GraphWorld may struggle in extremely dense or unpredictable scenarios where interaction cues are ambiguous or rapidly changing, leading to potential inaccuracies in the latent state.
- The model's complexity and reliance on high-quality perception and map data impose computational and hardware constraints, limiting deployment in resource-constrained environments.
- Current training relies heavily on supervised signals and dataset diversity; real-world generalization to unseen scenarios remains an open challenge, requiring further robustness enhancements.
Future Work
Future directions include integrating multimodal perception (e.g., radar, V2X data) to enrich environment representations, and applying reinforcement learning to optimize long-term decision policies. Additionally, reducing model complexity and improving robustness in highly dynamic or adverse conditions will be key to real-world deployment.
AI Executive Summary
GraphWorld introduces a pioneering framework for long-horizon autonomous driving by leveraging latent environment models. Traditional end-to-end systems excel at short-term decision-making but falter when reasoning over extended interactions, especially in complex traffic scenarios. To address this, GraphWorld constructs a dynamically built Ego-Centric Interaction Graph (ECIG), which selectively models neighboring agents based on spatial proximity. This graph propagates relational cues via cross-attention, capturing critical interaction dynamics. Complementing this, a flow-based continuous environment evolution model—Flow-Matching—learns to simulate future states without error-prone autoregressive rollouts, providing a stable, long-term predictive backbone. The entire system is trained end-to-end with multi-stage supervision, including temporal consistency constraints, ensuring stable, meaningful latent representations. Extensive experiments on datasets like nuScenes, NAVSIM, and Bench2Drive demonstrate that GraphWorld significantly reduces collision rates—by 19.5% over prior methods—and improves planning speed by 13.3%. These results highlight its potential to enable safer, more reliable autonomous vehicles capable of operating in highly interactive environments. The approach not only advances the technical frontier of long-range planning but also opens new avenues for integrating environment understanding with decision-making, promising transformative impacts on autonomous driving technology.
Deep Analysis
Background
Autonomous driving技术经历了从感知到决策的逐步演变，早期依赖规则和模型预测，后续引入深度学习实现端到端训练。代表性工作如UniAD、VAD、DiffusionDrive推动了轨迹多模态生成和决策优化，但普遍受限于短视能力。世界模型的出现为理解环境动态提供新思路，强调抽象表达与环境理解，逐渐成为长远规划的重要方向。近年来，潜在世界模型逐渐崛起，强调抽象表达与环境理解，成为长远规划的重要方向。尽管如此，将潜在模型与端到端架构结合，仍面临信息融合、动态建模和效率的挑战。
Core Problem
当前端到端自主驾驶系统多集中于短期轨迹预测，缺乏对长远交互和动态演变的建模能力。这导致在复杂、多智能体环境中，系统容易出现安全风险和决策失误。尤其是在多车交汇、突发事件频发的场景下，短视模型难以提前预判潜在冲突，限制了自主系统的安全性和泛化能力。如何在保证实时性前提下，有效捕获环境的长远动态信息，成为核心难题。传统方法多采用多步预测或像素生成，计算成本高，误差累积严重，难以满足实际应用需求。
Innovation
本研究提出GraphWorld框架，创新点在于：1）引入动态自我中心交互图（ECIG），通过邻居筛选和关系传播，有效捕获关键交互信息；2）利用Flow-Matching学习连续潜在世界演变，避免误差累积，提升长远推理稳定性；3）结合多阶段训练和时间监督，增强潜在状态的时间一致性。该方法突破了传统短视模型的局限，实现了高效、稳定的长距离规划，兼顾安全性和实时性。其核心创新在于关系图的动态构建和连续动态建模的结合，为多智能体环境中的长远决策提供了新思路。
Methodology
- �� 通过感知模块提取多智能体实例特征。
- �� 构建动态的ECIG，筛选邻居，利用交叉注意机制传播关系信息。
- �� 利用GRU编码历史动态和地图信息，形成时间一致的潜在世界状态。
- �� 采用Flow-Matching学习连续世界演变，定义当前与目标状态间的插值路径。
- �� 通过多模态轨迹解码头输出多样化轨迹，结合重要性重加权优化预测。
- �� 引入时间一致性监督，确保潜在状态的动态演变符合真实环境。
- �� 端到端训练结合感知、预测和规划损失，提升整体性能。
Experiments
在nuScenes、NAVSIM和Bench2Drive等多个公开数据集上进行评估，采用碰撞率、轨迹误差和规划距离作为指标。对比多种SOTA方法，包括World4Drive、DiffusionDrive等，验证GraphWorld在长距离规划中的优越性。通过消融实验分析ECIG邻居筛选和Flow-Matching的贡献。调优超参数如邻居数K、潜在状态维度和训练阶段，确保模型在复杂场景中的鲁棒性和实时性。
Results
GraphWorld在nuScenes 6秒长远规划中，将平均碰撞率降低19.5%，在NAVSIM上速度提升13.3%，优于现有多智能体模型。在复杂交互环境中，误差明显低于传统短视模型，验证了潜在世界模型在安全性和鲁棒性上的优势。消融实验显示，ECIG的邻居筛选和Flow-Matching的连续建模是性能提升的关键因素。整体表现证明该方法在长远规划中的实用性和优越性。
Applications
可应用于自动驾驶车辆的长距离路径规划、复杂交互环境中的安全决策，以及多智能体协作场景。模型依赖高质量感知和地图信息，适合未来智能交通系统的核心技术基础。其高效性和鲁棒性，有望推动自动驾驶在城市复杂环境中的部署，提升交通安全和效率。
Limitations & Outlook
模型在极端复杂场景下仍存在误判风险，尤其在多智能体密集交互或突发事件中表现不足。训练成本较高，硬件要求较大，实际部署中需优化模型复杂度。未来需增强模型对极端情况的适应性，并降低计算资源需求，以实现更广泛的应用。
Plain Language Accessible to non-experts
想象你在一个繁忙的厨房里做饭。每次你都要考虑不同的食材、厨具和烹饪步骤，甚至要预测未来可能发生的变化，比如火候变大或食材变少。传统的厨师可能只关注眼前的事情，做完一道菜就结束了，但有经验的厨师会提前考虑接下来几步的变化，确保菜肴最终完美。GraphWorld就像这个有经验的厨师，它用一种特殊的“脑袋”——潜在世界模型，提前模拟未来的变化，帮助驾驶汽车在复杂的交通环境中做出更安全、更长远的决策。它通过观察周围的“邻居”车辆，建立关系图，然后用数学方法预测未来的交通状况，确保每一步都稳妥可靠。这就像在厨房里提前想好所有可能出现的情况，确保菜肴不糊，也不缺料。这样，汽车就能像厨师一样，提前准备好下一步，避免事故，顺利到达目的地。
ELI14 Explained like you're 14
想象你在玩一个超级复杂的模拟游戏，你需要控制一辆车在繁忙的街道上跑。普通的游戏AI可能只会看眼前的路况，快速做出反应，但不能提前预料未来的危险。GraphWorld就像一个聪明的助手，它能提前“想象”未来几秒钟会发生什么，比如前面那辆车会突然变道或者红灯亮起。它用一种特别的“脑袋”——潜在世界模型，来模拟未来的交通场景。它会观察周围的车辆和道路，把这些信息变成一张关系图，就像在脑海里画出一张交通地图，然后用数学方法预测未来的变化。这样，汽车可以提前做出反应，避免碰撞，就像你提前知道下一步会发生什么一样。这个技术让自动驾驶变得更聪明、更安全，就像你在游戏中用策略赢得比赛一样。未来，这样的技术还能帮汽车在复杂环境中自主决策，变得更像人类司机一样聪明和可靠。
Abstract
End-to-end autonomous driving has made significant progress by unifying perception, prediction, and planning within a single learning framework, achieving strong performance in short-horizon decision making. However, most existing E2E-AD methods remain confined to short-horizon planning and lack the ability to model long-term temporal dependencies, which severely limits their generalization and security in complex and highly interactive driving scenarios. In this work, we propose GraphWorld, an E2E-AD framework that explicitly enhances long-horizon planning through latent world modeling. We introduce an Ego-Centric Interaction Graph, which adaptively models critical neighboring agents based on spatial proximity, and propagates relational context to planning queries via cross-node cross-attention. We present a World-State-Conditioned Planning that learns ego-centric latent world representations by modeling interactions between an ego vehicle and surrounding agents. This latent world state captures key interaction dynamics and safety-relevant semantics, and serves as a conditioning signal to guide long-horizon, safety-aware trajectory planning. Extensive experiments on Bench2Drive, NAVSIMv1/2, and nuScenes demonstrate that GraphWorld significantly reduces collision rates and improves long-horizon planning performance, validating its effectiveness in complex driving environments.