Req2Road: A GenAI Pipeline for SDV Test Artifact Generation and On-Vehicle Execution
Req2Road employs LLMs and VLMs with retrieval-augmented generation to automate SDV test artifact creation and vehicle execution, boosting testing efficiency.
Key Findings
Methodology
The approach integrates retrieval-augmented generation (RAG) with models like GPT-4 and Qwen 2.5 VL to extract signals and behaviors from heterogeneous artifacts. The pipeline includes initial Gherkin scenario creation, VSS signal mapping, scenario refinement, and code generation, ensuring semantic and syntactic correctness. Standardized VSS references enhance portability across systems. Experiments on Child Presence Detection validate the pipeline, covering Gherkin validity, mapping accuracy, and end-to-end execution in virtual and real vehicle environments.
Key Results
- 89% of the 36 requirements (32/36) were successfully transformed into executable scenarios, demonstrating high automation and efficiency. VSS mapping accuracy reached 100% in small candidate pools (16 signals), with slight degradation in full catalogs, outperforming manual efforts. Both virtual and real vehicle tests confirmed script stability and correctness, reducing manual intervention significantly.
- Models showed strong robustness in VSS mapping, especially with limited candidate pools, validating the effectiveness of pre-filtering strategies. GPT-4.1 achieved 100% pass rate in complex code generation tasks, indicating promising automation potential.
- Overall, the architecture markedly advances requirements-to-test automation, reducing manual workload and enabling large-scale validation, thus supporting safety-critical automotive applications.
Significance
This work bridges the gap between natural language requirements and executable tests by leveraging multimodal models and standard signal schemas. It addresses longstanding challenges in automotive validation—such as heterogeneous artifacts, low automation, and poor traceability—by providing a scalable, standardized, end-to-end pipeline. The approach enhances validation coverage, reduces human errors, and accelerates safety certification processes, contributing significantly to the deployment of autonomous driving systems and intelligent vehicle functions.
Technical Contribution
The main innovations include the integration of retrieval-augmented signal filtering, multimodal semantic alignment, and standardized VSS referencing within an end-to-end framework. This combination improves signal mapping accuracy, reduces hallucinations, and simplifies cross-system compatibility. The pipeline's modular design enables scalable deployment across different vehicle platforms, pushing the frontier of AI-driven automotive testing and validation.
Novelty
This is the first comprehensive framework that combines multimodal AI models with retrieval mechanisms to automate the entire requirements-to-test process in automotive systems. Unlike prior work focusing on low-level script generation or model-based test derivation, this approach directly translates heterogeneous artifacts into standardized, executable scenarios, significantly reducing manual effort and increasing flexibility. Its novelty lies in the seamless integration of standards, multimodal understanding, and retrieval strategies for scalable automotive testing.
Limitations
- Model performance diminishes with increasing complexity of requirements, especially when logical conditions are intricate or poorly specified, necessitating manual review.
- Large-scale signal libraries pose retrieval and processing challenges, requiring further optimization for real-time performance.
- Fully autonomous validation remains limited; human oversight is still needed for scenario refinement and edge cases.
Future Work
Future research will focus on enhancing model robustness in complex logical scenarios, optimizing retrieval algorithms for larger signal databases, and integrating reinforcement learning to improve signal relevance filtering. Additionally, expanding the pipeline to cover more vehicle subsystems and incorporating real-time feedback will be key to industrial deployment and continuous validation workflows.
AI Executive Summary
The rapid evolution of autonomous vehicle technology demands more efficient and reliable validation methods. Traditional testing approaches, heavily reliant on manual effort, struggle to keep pace with increasing complexity and regulatory requirements. Existing solutions often depend on structured models or low-level scripts, which are labor-intensive and lack flexibility.
This paper introduces Req2Road, an innovative end-to-end pipeline leveraging large-scale language models (LLMs) and multimodal vision-language models (VLMs) to automate the transformation of heterogeneous requirements artifacts into executable test scenarios. Central to the approach is the use of retrieval-augmented generation (RAG), which pre-filters relevant vehicle signals from vast signal repositories, reducing hallucinations and improving mapping accuracy. The pipeline encompasses four stages: initial scenario generation, VSS signal mapping, scenario refinement, and code synthesis, all designed to ensure semantic consistency and technical correctness.
Experimental validation on a Child Presence Detection system demonstrates the pipeline's effectiveness. Results show that 89% of the requirements could be directly converted into executable Gherkin scenarios without manual modification. The models achieved near-perfect signal mapping accuracy in small candidate pools and maintained high code correctness in complex scenarios. Both virtual and real vehicle tests confirmed the stability and reliability of the generated test scripts, significantly reducing manual effort and increasing validation throughput.
This work marks a significant step toward scalable, automated safety validation in automotive systems. By standardizing signal references via VSS and enabling end-to-end automation, it addresses key industry challenges—heterogeneous artifacts, low automation levels, and traceability gaps. The approach paves the way for future integration with continuous validation workflows, reinforcement learning-based signal filtering, and broader subsystem coverage, ultimately accelerating the deployment of safe autonomous vehicles.
Deep Analysis
Background
随着自动驾驶和智能车辆技术的快速发展，车辆软件的复杂性不断上升，传统验证方法已难以满足高效、全面的安全保障需求。早期研究如UML模型驱动测试（Shin和Lim）在结构化模型基础上实现自动化，但难以应对需求多样性和异构工件。近年来，AI辅助的测试生成逐渐兴起，例如Wynn-Williams利用AI转化需求，Karlsson用Copilot生成测试脚本，显著提升了自动化水平。然而，这些方法多依赖特定工具链，缺乏从需求到测试的完整自动化流程。本研究结合多模态模型和VSS标准，提出了端到端的需求转测试架构，旨在解决多源异构工件到可执行测试的断层，推动工业自动化验证的发展。
Core Problem
现有方法在需求理解、信号映射和测试脚本自动生成方面存在瓶颈。需求描述繁杂且模糊，难以直接转化为测试场景。信号定义不统一，导致跨工具链迁移困难。手工编写测试脚本耗时长，易出错，难以满足大规模验证需求。如何实现从自然语言需求到高质量、可执行测试的自动化，是行业亟待解决的问题。解决方案需兼顾多源异构需求、信号标准化和自动化效率，具有重要的实际意义。
Innovation
本研究的创新点包括：1）引入检索增强生成（RAG）机制，有效筛选相关VSS信号，减少模型幻觉；2）结合多模态模型实现需求文本与信号的语义对齐，提升映射准确率；3）采用标准化VSS信号体系，解决异构工具链间的兼容性问题；4）实现从自然语言需求到Gherkin场景、信号映射、测试代码的全流程自动化。这一架构突破了传统依赖结构化模型的限制，简化了流程，提升了自动化水平，为工业应用提供了可扩展的解决方案。
Methodology
- �� 输入：自然语言需求、UML图和流程图。• 需求解析：利用LLMs理解需求内容，提取关键行为。• 初步场景生成：基于需求自动生成Gherkin场景，提供修正空间。• 信号筛选：使用SentenceTransformer嵌入，结合RAG机制，从VSS库中筛选候选信号。• 信号映射：利用LLMs结合候选池，选择最相关的VSS路径，减少幻觉。• 场景优化：插入映射信号，确保语义一致。• 代码生成：用GPT-4.1根据场景自动生成Python测试脚本。• 验证：在虚拟环境和实车上执行，确保端到端可用性。
Experiments
在Child Presence Detection系统上，使用36个需求进行验证。模型在不同候选池（16信号与全库）下评估映射准确率，比较GPT-4o-mini与本地Vicuna模型。指标包括Gherkin有效性、映射正确率和代码执行成功率。虚拟环境和实车验证确保脚本稳定性。通过不同模型和参数的对比，验证预筛选策略的有效性。整体设计旨在评估端到端流程的自动化和实用性，为工业化应用提供依据。
Results
模型在信号映射中表现优异，16信号候选池中，GPT-4o-mini实现100%匹配（4/4），全库中表现略有下降但仍优于手工方法。自动生成的Gherkin场景中，89%的需求无需修改即可执行。代码生成方面，GPT-4.1在复杂场景中达到了100%的通过率。虚拟和实车测试验证了脚本的稳定性，显著减少了人工干预。模型在处理边界条件和复杂逻辑时表现出一定局限，但整体验证了端到端自动化的可行性。
Applications
该架构可广泛应用于自动驾驶、智能座舱等安全关键系统的验证流程，支持多源异构需求的快速转化，提升验证效率。企业可利用此流程实现大规模自动化测试，降低成本并提升安全性。未来可结合持续集成（CI）系统，实现实时需求变更的快速验证，为产业化提供技术支撑。
Limitations & Outlook
模型在处理复杂逻辑和多条件组合时仍存在误差，特别在需求描述模糊或边界条件未明确时表现不佳。信号库规模扩大后，检索性能成为瓶颈。自动生成的脚本在极端场景下仍需人工验证，完全自动化尚未实现。未来需优化模型鲁棒性和检索效率，增强系统的适应性和可靠性。
Plain Language Accessible to non-experts
想象你在厨房做饭，需求就像食谱，告诉你要做什么菜。传统的方法是你自己一字一句写食谱，容易出错，还得反复试验。而现在，有个聪明的助手（AI），它能看懂你的需求，帮你自动写出详细的做菜步骤，还能根据厨房里的材料（信号）帮你调整菜谱。这个助手还能帮你检查每一步是否合理，确保菜能做好。这样一来，从需求到实际操作，只需几秒钟，省时又省力。就像有个超级厨师帮你搞定所有细节，让厨房变得更智能、更高效。
ELI14 Explained like you're 14
想象你在学校里，有个超级聪明的朋友，他能帮你写作业、做实验，还能告诉你答案。你只要告诉他你遇到的问题，他就能用自己的知识帮你解决。比如，你要做一个科学项目，他会帮你设计步骤，告诉你需要用什么材料，怎么操作。这个朋友还会检查你的每一步，确保没有错。以前你得自己花很多时间查资料、写步骤，现在有了这个聪明的朋友，一切变得简单多了。它就像一个超级助手，让你学习和做事都变得更快、更准确。
Glossary
Gherkin (Gherkin syntax)
A structured language used in Behavior-Driven Development (BDD) to describe test scenarios in Given/When/Then format, making tests understandable for non-technical stakeholders.
Used to formalize test scenarios in the pipeline.
VSS (Vehicle Signal Specification)
A standardized hierarchical schema defining vehicle signals and APIs, enabling signal reference consistency and cross-system compatibility.
Ensures signal references are standardized and portable.
Retrieval-Augmented Generation (RAG)
A hybrid approach combining information retrieval with generative models to improve relevance and accuracy of generated outputs.
Used for preselecting relevant signals from large VSS repositories.
Multimodal Models
Deep learning models that process and fuse multiple data modalities, such as text and images, for comprehensive scene understanding.
Facilitate semantic alignment between textual requirements and visual diagrams.
End-to-End Automation
A process where input requirements are directly transformed into executable tests without manual intervention, ensuring seamless workflow.
Core goal of the Req2Road pipeline.
Open Questions Unanswered questions from this research
- 1 如何进一步提升模型在多条件、多信号交叉场景中的映射准确率，尤其在需求描述模糊或边界条件未明确时，仍需探索更鲁棒的多模态融合和推理机制。
Applications
Immediate Applications
自动化验证流程
企业可利用该架构实现需求到测试的快速自动化，提升验证效率，降低人工成本，特别适用于自动驾驶安全验证。
快速需求变更验证
支持在需求调整后，快速生成对应测试场景，缩短验证周期，增强敏捷开发能力。
Long-term Vision
全自动安全验证体系
结合持续集成平台，推动自动化验证流程的全面落地，实现实时需求变更的快速验证，助力产业规模化应用。
Abstract
Testing functionality in Software-Defined Vehicles is challenging because requirements are written in natural language, specifications combine text, tables, and diagrams, while test assets are scattered across heterogeneous toolchains. Large Language Models and Vision-Language Models are used to extract signals and behavioral logic to automatically generate Gherkin scenarios, which are then converted into runnable test scripts. The Vehicle Signal Specification (VSS) integration standardizes signal references, supporting portability across subsystems and test benches. The pipeline uses retrieval-augmented generation to preselect candidate VSS signals before mapping. We evaluate the approach on the safety-relevant Child Presence Detection System, executing the generated tests in a virtual environment and on an actual vehicle. Our evaluation covers Gherkin validity, VSS mapping quality, and end-to-end executability. Results show that 32 of 36 requirements (89\%) can be transformed into executable scenarios in our setting, while human review and targeted substitutions remain necessary. This paper is a feasibility and architectural demonstration of an end-to-end requirements-to-test pipeline for SDV subsystems, evaluated on a CPDS case in simulation and Vehicle-in-the-Loop settings.