A Low-Cost, Open Platform for End-to-End Autonomous Driving on a Miniature Ackermann Vehicle

Key Findings

Methodology

The platform integrates a physical vehicle, printed urban track, data collection pipeline, trajectory registration, and Webots digital twin. It employs command-conditioned behavior cloning, where a neural network receives onboard camera images and high-level navigation commands, producing steering and speed outputs. Experiments validate the approach both physically and in simulation. In real closed-loop tests, the learned policy maintains lane following and executes turns with a mean cross-track error of 6.1cm, close to human demonstrations at 4.7cm. The digital twin analysis shows that increasing camera field of view from 58° to 120° reduces error from 35.6cm to 3.3cm. Synthetic data generated in simulation, combined with a learned image translator to bridge appearance gaps, enables training higher-capacity policies that successfully complete all four routes in closed loop, outperforming models trained solely on real data.

Key Results

In physical experiments, the policy achieved a mean cross-track error of 6.1cm, comparable to human performance at 4.7cm. Widening camera FOV from 58° to 120° dramatically reduced error from 35.6cm to 3.3cm, highlighting the importance of perception range. Synthetic data augmentation with image translation enabled high-capacity models to complete all four routes, with errors around 5cm, surpassing real-only models. Ablation studies confirmed that larger models and synthetic data improve robustness and generalization.
The digital twin experiments demonstrated that increasing camera FOV significantly enhances lane following accuracy. The combined synthetic and real data training yielded models capable of full route completion, with mean errors below 6cm across different routes. These results validate the effectiveness of the proposed sim-to-real pipeline and the importance of perception parameters.
Overall, the platform provides a practical, low-cost testbed for end-to-end autonomous driving, facilitating research on transfer learning, perception, and control. The open-source release supports reproducibility and community engagement, accelerating progress toward real-world deployment.

Significance

This work bridges the gap between high-fidelity simulation and real-world autonomous driving using a low-cost, open platform. It demonstrates that with appropriate perception enhancements and synthetic data augmentation, deep learning policies can reliably transfer from simulation to physical hardware. The platform enables systematic evaluation of perception and control strategies under real constraints, fostering reproducible research. Such accessible tools democratize autonomous driving research, lowering barriers for academia and industry, and accelerating innovation in safe, scalable self-driving systems.

Technical Contribution

The study introduces a comprehensive, low-cost hardware and software framework combining physical vehicles, digital twins, and synthetic data. It innovatively analyzes the impact of camera field of view on path following, and employs a learned image translation to reduce sim-to-real appearance gaps. The use of command-conditioned behavior cloning enhances policy robustness across diverse maneuvers. The open-source design and detailed ablation studies provide valuable insights into perception, model capacity, and data augmentation effects, advancing the state-of-the-art in end-to-end autonomous driving research.

Novelty

This is the first demonstration of end-to-end autonomous driving on a low-cost miniature Ackermann vehicle with an integrated digital twin and synthetic data augmentation. The systematic analysis of camera field of view's effect on performance and the application of learned image translation for sim-to-real transfer are novel contributions. Unlike traditional high-cost platforms, this setup emphasizes accessibility, reproducibility, and practical deployment, offering a new paradigm for scalable autonomous driving research.

Limitations

Limited number of physical runs restricts statistical robustness; larger datasets are needed for broader validation.
Manual trajectory registration introduces subjective errors, impacting quantitative evaluation accuracy.
Model complexity is constrained by hardware, limiting scalability to more sophisticated architectures or multi-sensor fusion.
Current experiments focus on simple urban tracks; performance in complex, dynamic environments remains untested.

Future Work

Future efforts will focus on automating trajectory registration, expanding perception range with wider FOV cameras, and developing more advanced models such as BEV-based or multimodal architectures. Integration of additional sensors and reinforcement learning techniques could further improve robustness. Hardware upgrades for faster inference and real-time processing are planned, aiming toward deployment in more complex, real-world scenarios.

AI Executive Summary

Deep Dive

Plain Language Accessible to non-experts

想象你在玩一款遥控车游戏，但这辆车还能自己学会开路。你只需要告诉它要走哪条路，比如直行或转弯，然后它会用摄像头观察前方的路面，自己调整方向。科学家们用一种叫行为克隆的方法，让车模仿人类驾驶的样子，教它怎么转弯、保持车道。为了让车更聪明，研究人员还在电脑模拟的城市里练习，让车在虚拟环境中学习，然后把学到的技能带到真实的小车上。这样，既安全又省钱，还能不断改进。

ELI14 Explained like you're 14

想象你在玩一款赛车游戏，你可以用手柄控制赛车，但如果你让电脑学会模仿你的驾驶方式，它就能自己开车了！科学家们做的事情也是一样，他们用一辆小车和一台电脑，让它看着路、听着指令，然后学会自己开车。为了让小车更聪明，他们还在电脑模拟的城市里练习，之后把学到的技能带到真实的小车上。这样，小车就能在真实的道路上自己跑，跟人一样准。这就像教会一只宠物学会跑步和转弯一样，既安全又有趣！

Glossary

Behavior Cloning (行为克隆)

一种模仿人类驾驶行为的学习方法，通过模仿示范数据训练模型，使其学会自主驾驶。技术上基于监督学习，映射感知到动作。

论文中采用条件行为克隆，输入摄像头图像和导航指令，输出转向和速度。

Digital Twin (数字孪平台)

虚拟环境中对实体车辆和轨道的高精度模拟，用于测试和验证自主驾驶策略。结合仿真和实车数据，支持迁移学习。

用Webots模拟轨道和车辆，生成合成数据，验证模型性能。

Sim-to-Real Transfer (仿真到实车迁移)

将在仿真环境中训练的模型迁移到真实硬件上，解决感知差异、感知噪声和控制误差带来的挑战。

通过图像翻译缩小仿真与实车外观差异，提升迁移效果。

View Field of View (视场角)

摄像头能观察到的最大角度范围，影响感知能力和路径跟踪精度。

扩大视场角从58°到120°，显著降低横向误差。

Conditional Behavior Cloning (条件行为克隆)

在模型输入中加入高层导航指令，解决交叉路口等多样场景下的模糊问题。

模型根据指令选择转弯或直行，提高多样性和鲁棒性。

Open Questions Unanswered questions from this research

1 当前平台在复杂交通环境中的表现尚未验证，尤其在多车交互和动态障碍下的鲁棒性不足。
2 自动轨迹注册依赖人工标注，未来需开发自动化、精确的注册算法以提升评估效率。
3 模型在多模态感知融合方面仍有提升空间，结合雷达、激光等传感器可能带来更强的环境理解能力。

Applications

Immediate Applications

自主驾驶算法验证平台

科研和工业界可利用该平台快速验证端到端深度学习模型，降低研发成本，提升算法迁移效率。

教育与培训工具

作为自动驾驶基础课程的实验平台，帮助学生和工程师理解感知、控制与迁移学习的流程。

Long-term Vision

自动驾驶系统的标准化测试环境

推动建立统一的评估体系，促进不同算法和硬件平台的公平比较，加速产业化落地。

Abstract

This paper presents a low-cost, open experimental platform for research in end-to-end autonomous driving with miniature Ackermann vehicles. The platform combines a physical vehicle, a printed urban track, data collection tools, trajectory registration, and a Webots digital twin, enabling controlled experiments that connect simulation-based autonomous-driving methods to real-world execution. As a first baseline, we implement command-conditioned behavior cloning, in which a neural policy receives an on-board camera image and a high-level navigation command and outputs steering and speed. The system is evaluated both on the physical vehicle and in simulation. In real closed-loop experiments, the learned policy follows lanes and executes commanded turns, reaching a mean cross-track error of 6.1 cm with respect to the reference route, close to the 4.7 cm observed in human demonstrations. In the digital twin, camera field of view has a strong effect on performance, reducing the mean cross-track error from 35.6 to 3.3 cm when widened from 58 to 120 degrees. Using the digital twin to generate synthetic driving data and a learned sim-to-real image translator to reduce the appearance gap, we further show that a higher-capacity policy trained on this synthetic data combined with real demonstrations is the only configuration that completes all four track routes in closed loop, whereas the compact baseline and the same network trained on real data alone complete fewer. These results establish the open platform as a practical testbed for sim-to-real studies and provide an initial command-conditioned imitation-learning baseline; we release it to support reproducible research.

cs.LG cs.AI cs.RO