Collaborative Dynamic 3D Scene Graphs for Automated Driving
Proposes CURB-SG, a multi-agent LiDAR-based urban 3D scene graph for enhanced environment understanding in autonomous driving.
Key Findings
Methodology
This paper introduces a hierarchical 3D scene graph framework based on collaborative SLAM, integrating panoptic LiDAR data from multiple agents. The system detects inter-agent loop closures using ICP-based matching, optimizes a global pose graph via g2o, and employs edge contraction for scalability. It constructs a lane graph from vehicle trajectories and observations, partitioning the environment into intersections and roads. The multi-layer scene graph combines lane, static landmarks, dynamic vehicles, and pose information, enabling efficient spatial and semantic queries. The approach leverages the CARLA simulator for validation, demonstrating significant improvements in map accuracy, exploration speed, and scene completeness across multi-agent scenarios.
Key Results
- In simulated urban scenarios, the multi-agent system reduced localization RMSE from 0.735m (single agent) to 0.132m (three agents) in Town01, with a 30% faster exploration rate. Lane graph IoU increased from 0.81 to 0.92, indicating better environment coverage. The pose graph node count was effectively controlled via edge contraction, enabling frequent optimization. The environment partitioning accurately identified intersections and road segments, supporting path planning and dynamic object tracking.
- The edge contraction mechanism significantly limited pose graph growth, maintaining real-time optimization feasibility. Multi-agent cooperation improved map completeness and robustness, especially in dynamic traffic conditions. The lane graph achieved high spatial accuracy, covering most road networks and supporting complex intersection analysis, crucial for path planning.
- Overall, the framework demonstrated scalable, accurate, and dynamic environment modeling, with potential for real-world deployment in urban autonomous vehicles, enhancing safety and efficiency.
Significance
This work advances urban environment perception by integrating multi-agent cooperation with hierarchical scene graphs, addressing the scalability and dynamicity challenges of city-scale mapping. It provides a comprehensive, queryable environment model that supports perception, planning, and decision-making in complex traffic scenarios. The approach bridges the gap between indoor scene graph methods and outdoor urban environments, offering a scalable solution for real-time, semantic-rich city mapping. Its ability to continuously update and refine maps with dynamic traffic participants makes it highly relevant for future autonomous driving systems, enabling safer and more reliable navigation in dense urban areas.
Technical Contribution
The paper introduces a novel multi-agent collaborative SLAM pipeline combined with a hierarchical scene graph structure tailored for urban environments. Key innovations include: (1) a panoptic LiDAR data fusion method for large-scale mapping; (2) an inter-agent loop closure detection mechanism for global consistency; (3) an edge contraction technique to manage pose graph scalability; (4) a lane graph construction from vehicle trajectories and observations; (5) a multi-layer environment partitioning strategy based on connectivity and intersections. These contributions enable scalable, accurate, and dynamic urban environment modeling, supporting complex queries and reasoning.
Novelty
This is the first work to tightly couple multi-agent collaborative SLAM with hierarchical urban scene graphs, leveraging cross-agent loop closures for consistent large-scale maps. Unlike indoor scene graph methods, it introduces environment partitioning based on a lane graph, suitable for complex city topologies. The integration of edge contraction for scalability and the multi-layered environment abstraction are novel, addressing the unique challenges of urban autonomous driving.
Limitations
- Dependence on high-quality panoramic LiDAR data makes the system vulnerable under adverse weather or occlusion conditions, potentially degrading performance.
- Computational complexity increases with environment size and number of agents, posing challenges for real-time deployment without hardware acceleration.
- Dynamic object modeling remains coarse, requiring future integration of predictive models for better traffic understanding.
Future Work
Future research will focus on integrating multi-modal sensors (camera, radar) to improve perception robustness, especially under challenging weather. Efforts will also target distributed architectures to enhance scalability and real-time performance. Additionally, incorporating predictive dynamic scene understanding and deep learning-based semantic reasoning will further refine environment modeling, paving the way for fully autonomous urban navigation.
AI Executive Summary
Autonomous driving in urban environments demands precise, scalable, and dynamic scene understanding. Traditional mapping approaches, often relying on manual annotations or limited sensor modalities, struggle to meet the complexity of cityscapes. To address this, the proposed CURB-SG framework combines multi-agent collaborative SLAM with hierarchical scene graphs, enabling comprehensive environment modeling. Multiple vehicles equipped with panoramic LiDAR share data with a central server, which detects inter-agent loop closures using ICP-based matching, and performs global pose graph optimization via g2o. To manage large-scale maps, edge contraction reduces graph complexity, facilitating frequent updates. The system constructs a lane graph from vehicle trajectories and observations, partitioning the environment into intersections and roads, and integrates static landmarks and dynamic vehicles into a multi-layer scene graph. This layered structure supports efficient spatial and semantic queries, crucial for path planning and obstacle avoidance. Extensive experiments in CARLA simulations demonstrate that multi-agent cooperation improves localization accuracy, map completeness, and exploration speed by over 30%. The lane graph achieves IoU scores exceeding 0.81, accurately representing complex urban road networks. These results validate the framework’s scalability, robustness, and real-time capability, marking a significant step toward practical urban autonomous driving. Despite current limitations in adverse weather robustness and dynamic object modeling, future work aims to incorporate multi-modal perception, distributed processing, and predictive scene understanding, promising a transformative impact on city-scale autonomous navigation.
Deep Analysis
Background
Urban environment perception一直是自动驾驶研究的核心难题。早期依赖人工标注的高清地图逐步被SLAM技术取代，如LOAM、HDL Graph SLAM等，显著提升了地图自动化和精度。近年来，结合语义信息的地图（SUMA++、C-LOAM）增强了环境理解能力。多智能体SLAM的出现，解决了大规模环境下地图构建的瓶颈，通过多车协作实现更快的覆盖与更新。场景图作为一种结构化环境表示，已在室内场景中取得成功，但在城市环境中应用仍有限。本文将多智能体SLAM与层次化场景图结合，填补了城市环境大规模、动态、多模态感知的研究空白，为自动驾驶提供更全面的环境认知基础。
Core Problem
城市环境复杂多变，单一智能体难以实现全面、实时的场景理解。传统SLAM面临地图规模扩大带来的计算瓶颈，缺乏高效的环境语义抽象。多智能体合作虽能缓解部分问题，但跨智能体回环检测与全局优化仍存在挑战。现有方法多侧重静态环境，动态交通参与者的建模不足，限制了自动驾驶的安全性与可靠性。如何融合多源感知信息，构建高效、动态、层次化的城市场景图，成为亟待解决的核心问题。
Innovation
本研究的创新点包括：1）提出多智能体合作SLAM框架，利用全景LiDAR数据实现全局一致性，突破单车局限；2）引入边缘收缩技术，有效控制pose图规模，确保实时优化；3）基于车道连接关系，自动划分交叉口与道路区域，增强环境空间理解；4）构建多层次场景图，融合静态地标与动态车辆信息，支持复杂场景的空间与语义推理。这些创新共同推动城市级大规模环境的高效建模与理解，为自动驾驶环境感知提供了全新解决方案。
Methodology
- �� 多智能体LiDAR数据采集：每个车辆配备全景LiDAR，进行点云捕获与语义分割，区分静态与动态对象。
- �� 本地配准：采用FAST-GICP算法进行连续点云配准，估算车辆运动。
- �� 关键帧生成：基于行驶距离，定期传输静态点云与位姿到中心服务器。
- �� 全局图优化：在服务器端，结合跨智能体回环检测（ICP匹配得分阈值）进行姿态图优化（g2o），利用边缘收缩控制图规模。
- �� 跨智能体回环检测：利用点云描述子识别重叠区域，确保地图一致性。
- �� 车道图构建：提取车辆轨迹与观察到的车辆位置，构建有向车道图，划分交叉口与非交叉区。
- �� 场景图层次化：将车道、静态地标、动态车辆、SLAM姿态图融合，形成多层次环境表示，支持空间与语义查询。
Experiments
采用CARLA模拟器进行多场景、多智能体测试，验证地图构建、定位精度与环境划分效果。比较不同智能体数量对地图完整性与构建速度的影响，分析边缘收缩对系统性能的优化作用。指标包括：RMSE定位误差、地图IoU、探索速度、pose图节点数等。通过模拟多次回环与动态交通，评估系统的鲁棒性与实时性。实验还验证了车道图的空间覆盖率与划分准确性，为路径规划提供支持。
Results
多智能体合作显著降低定位误差（如Town01中由0.735米降至0.132米），地图构建速度提升约30%，IoU指标由0.81提升至0.92。边缘收缩有效控制pose图节点数，确保频繁优化成为可能。车道图覆盖大部分道路网络，IoU达0.81，支持复杂交叉口的空间划分。动态对象识别与静态地标定位准确，为自动驾驶路径规划提供可靠基础。这些结果验证了多智能体合作与层次化场景图的有效性。
Applications
该框架适用于城市自动驾驶环境的实时地图构建与环境理解，支持路径规划、动态避障与交通管理。通过多智能体协作，可大幅提升大规模环境的感知效率与准确性，适合未来自动驾驶车辆的集群部署。未来结合深度学习与多模态感知，将进一步增强系统的环境适应能力与鲁棒性。
Limitations & Outlook
当前方法对传感器数据质量敏感，极端天气或遮挡条件下表现不佳。系统在大规模、多智能体环境中计算成本较高，实时性仍需优化。动态对象建模较为粗糙，未来需引入更细粒度的动态场景理解。
Plain Language Accessible to non-experts
想象你在一个大型工厂工作，工厂里有许多工人（车辆）在不同区域忙碌。每个工人都带着一台特殊的相机（LiDAR），可以看到周围的东西，还能记住自己走过的路径。工厂的管理系统（中央服务器）会收集所有工人的信息，帮忙拼出整个工厂的布局。每个工人会告诉系统他们看到的道路、标志和其他工人。系统会把这些信息整理成一张大地图，把道路、交叉口、静止的标志和移动的工人都标出来。这样，工厂的每个角落都被详细记录，工人们也可以根据这张地图找到最好的路径。这个过程不断更新，确保地图始终准确，工人们可以安全高效地工作。这个比喻帮助理解，自动驾驶车辆就像工人，地图就像工厂布局，合作就像工人们共同努力，确保每个人都知道环境的全貌。
ELI14 Explained like you're 14
想象你在玩一个超级复杂的迷宫游戏，你的任务是找到出口。可是这个迷宫很大，有很多不同的道路和转弯，你一个人很难记住所有的路径。于是，你邀请了几个朋友一起帮忙，每个人都带着一台特殊的相机，可以拍下他们看到的道路和障碍物。每当一个朋友走到新的地方，他们就把这段路的照片和位置告诉你。你们把所有朋友的照片和路径拼在一起，慢慢画出整个迷宫的地图。你们还会标记出交叉口、死胡同和重要的标志，比如红色的灯或者大门。通过合作，你们可以更快、更准确地知道迷宫的布局。每次有人走过的路都被记录下来，地图也会不断更新。这样，即使迷宫变得更复杂，你们也能找到最好的路线。这就像自动驾驶汽车一样，它们和朋友（其他车辆）合作，利用传感器一起绘制城市的地图，确保每辆车都知道周围的环境，安全行驶。
Abstract
Maps have played an indispensable role in enabling safe and automated driving. Although there have been many advances on different fronts ranging from SLAM to semantics, building an actionable hierarchical semantic representation of urban dynamic scenes and processing information from multiple agents are still challenging problems. In this work, we present Collaborative URBan Scene Graphs (CURB-SG) that enable higher-order reasoning and efficient querying for many functions of automated driving. CURB-SG leverages panoptic LiDAR data from multiple agents to build large-scale maps using an effective graph-based collaborative SLAM approach that detects inter-agent loop closures. To semantically decompose the obtained 3D map, we build a lane graph from the paths of ego agents and their panoptic observations of other vehicles. Based on the connectivity of the lane graph, we segregate the environment into intersecting and non-intersecting road areas. Subsequently, we construct a multi-layered scene graph that includes lane information, the position of static landmarks and their assignment to certain map sections, other vehicles observed by the ego agents, and the pose graph from SLAM including 3D panoptic point clouds. We extensively evaluate CURB-SG in urban scenarios using a photorealistic simulator. We release our code at http://curb.cs.uni-freiburg.de.