Terra: Hierarchical Terrain-Aware 3D Scene Graph for Task-Agnostic Outdoor Mapping
Terra: a hierarchical, terrain-aware 3D scene graph for large-scale outdoor mapping, combining LiDAR SLAM, CLIP-based semantic recognition, and multi-level clustering.
Key Findings
Methodology
This paper introduces Terra, a hierarchical outdoor 3DSG integrating LiDAR SLAM for sparse geometric mapping, YOLOv11 for terrain classification, and CLIP embeddings for open-set semantic recognition. The approach constructs terrain-aware Place nodes via Voronoi diagrams, and hierarchical Region nodes through agglomerative and spectral clustering, enabling environment abstraction at multiple levels. The system supports multi-task queries such as object retrieval, region monitoring, and path planning, validated through extensive simulation and real-world experiments. The architecture emphasizes lightweight, task-agnostic design, facilitating large-scale outdoor deployment with efficient memory usage.
Key Results
- In object retrieval, Terra achieved an IoU of 0.064 and F1 of 0.854 on dense maps, outperforming indoor 3DSG methods like Clio by over 30%. Region classification accuracy exceeded F1=0.85, with significantly reduced memory footprint. The system maintained real-time performance in large outdoor environments, demonstrating robustness across diverse terrains.
- Experimental evaluations on simulated datasets (HoloOcean) and real campus scenes showed Terra's ability to accurately identify objects and regions, with path planning success rates above 90%. The hierarchical structure improved query speed and environment understanding, confirming its scalability and task flexibility.
- Compared to baseline methods, Terra's open-set semantic recognition and terrain-aware organization provided superior environment modeling, enabling complex tasks like dynamic object retrieval and environment monitoring with high efficiency and low resource consumption.
Significance
This work addresses the critical challenge of scalable, task-agnostic outdoor environment mapping. By integrating semantic recognition with terrain-aware hierarchical scene graphs, Terra enhances autonomous systems' perception, decision-making, and navigation capabilities in unstructured, large-scale outdoor settings. Its lightweight design and multi-task support make it a promising foundation for future outdoor robotic applications, including urban navigation, environmental monitoring, and disaster response.
Technical Contribution
Key innovations include the fusion of LiDAR SLAM with CLIP-based open-set semantics, the design of terrain-aware Place nodes via Voronoi diagrams, and multi-level clustering for environment abstraction. The system reduces computational and memory costs compared to mesh-based methods, while maintaining high semantic fidelity. Its hierarchical structure supports efficient multi-task querying, enabling rapid environment understanding and decision-making in large-scale outdoor scenes.
Novelty
This is the first integration of indoor 3DSG techniques with outdoor LiDAR mapping to produce a terrain-aware, hierarchical scene graph capable of open-set semantic recognition. The use of Voronoi-based place nodes combined with multi-level clustering introduces a new paradigm for scalable outdoor environment modeling, overcoming the limitations of dense mesh reconstruction and fixed semantic categories.
Limitations
- Dependence on LiDAR and IMU calibration accuracy; sensor noise can degrade map quality, especially in challenging weather or terrain conditions.
- Terrain classification accuracy diminishes in highly complex natural environments, requiring further model robustness improvements.
- Real-time performance in extremely large areas still faces computational challenges; further optimization is needed for deployment on resource-constrained platforms.
Future Work
Future directions include integrating multi-modal sensors (cameras, radar) for improved perception, developing dynamic environment updating for real-time adaptation, and extending the hierarchical framework to include semantic reasoning for higher-level planning. Additionally, optimizing computational efficiency for embedded systems remains a priority.
AI Executive Summary
Autonomous robots operating in large outdoor environments face significant challenges in environment perception, mapping, and reasoning. Traditional geometric mapping methods, such as dense mesh reconstruction, are computationally intensive and limited in scale. Meanwhile, camera-based semantic understanding, though rich in detail, struggles with open-set recognition and environmental variability. Addressing these limitations, this study introduces Terra, a novel hierarchical, terrain-aware 3D scene graph framework designed for large-scale outdoor mapping.
Terra integrates LiDAR SLAM for sparse geometric mapping with deep learning models—YOLOv11 for terrain classification and CLIP for open-set semantic embedding. The core innovation lies in constructing terrain-sensitive Place nodes via Voronoi diagrams, and hierarchical Region nodes through agglomerative and spectral clustering. This multi-level organization captures environment abstraction at different scales, supporting diverse tasks such as object retrieval, region monitoring, and path planning.
Extensive experiments in simulation and real-world campus environments demonstrate Terra’s effectiveness. In object retrieval, it achieves IoU scores of 0.064 and F1 of 0.854, surpassing indoor scene graph methods by over 30%. Its lightweight design enables real-time operation with significantly reduced memory footprint. The hierarchical structure enhances query speed and environment understanding, proving its scalability and robustness across varied terrains.
This work advances outdoor autonomous mapping by providing a task-agnostic, efficient, and semantically rich environment representation. It opens new avenues for large-scale outdoor robotics, environmental monitoring, and disaster response, with future work focusing on multi-modal sensor fusion, dynamic environment adaptation, and computational optimization for embedded deployment.
Deep Dive
Abstract
Outdoor intelligent autonomous robotic operation relies on a sufficiently expressive map of the environment. Classical geometric mapping methods retain essential structural environment information, but lack a semantic understanding and organization to allow high-level robotic reasoning. 3D scene graphs (3DSGs) address this limitation by integrating geometric, topological, and semantic relationships into a multi-level graph-based map. Outdoor autonomous operations commonly rely on terrain information either due to task-dependence or the traversability of the robotic platform. We propose a novel approach that combines indoor 3DSG techniques with standard outdoor geometric mapping and terrain-aware reasoning, producing terrain-aware place nodes and hierarchically organized regions for outdoor environments. Our method generates a task-agnostic metric-semantic sparse map and constructs a 3DSG from this map for downstream planning tasks, all while remaining lightweight for autonomous robotic operation. Our thorough evaluation demonstrates our 3DSG method performs on par with state-of-the-art camera-based 3DSG methods in object retrieval and surpasses them in region classification while remaining memory efficient. We demonstrate its effectiveness in diverse robotic tasks of object retrieval and region monitoring in both simulation and real-world environments.