Sim2Real-AD: A Modular Sim-to-Real Framework for Deploying VLM-Guided Reinforcement Learning in Real-World Autonomous Driving

Recent advances in foundation models, such as large vision-language models (VLMs), have revolutionized perception and reasoning in autonomous systems. These models enable agents to learn complex, semantically grounded behaviors through reinforcement learning (RL), replacing brittle handcrafted rewards. Simulation environments like CARLA facilitate safe, scalable training, but the transition to real-world deployment faces significant hurdles. The core issues involve the observation domain gap—simulator-native structured data versus noisy, partial real sensor inputs—and the dynamics gap—discrepancies in vehicle responses due to physical differences. Prior work has addressed these separately via domain randomization, adversarial adaptation, or system identification, but none effectively unify the solution for foundation-model-guided RL policies. This gap remains a bottleneck for deploying advanced perception-control policies in real vehicles, especially without additional real-world data.

The main challenge is enabling zero-shot transfer of simulation-trained RL policies guided by VLMs onto real vehicles. The observation gap arises because real sensors produce partial, noisy images, whereas simulation provides complete, privileged BEV maps. The dynamics gap stems from differences in vehicle parameters like wheelbase, steering ratio, and actuator latency, which cause control commands to produce inconsistent vehicle responses. These two issues interact, causing policies trained in simulation to fail in real-world scenarios. Existing methods often address either gap separately, requiring extensive retraining or domain adaptation, which is costly and data-intensive. The problem is to develop a principled, modular approach that guarantees bounded transfer errors without additional real-world training.

The key innovations include: 1) Formalizing a transfer guarantee that decomposes the overall deployment error into three independent, controllable components—observation, dynamics, and task-geometry gaps—providing a theoretical foundation. 2) Designing the Geometric Observation Bridge (GOB) that employs segmentation and inverse perspective mapping to convert monocular images into structured BEV maps, reducing observation domain mismatch without training. 3) Developing the Physics-Aware Action Mapping (PAM) that translates RL outputs into platform-independent control commands using a bicycle model and PID controllers, decoupling policy learning from vehicle-specific dynamics. 4) Implementing a Two-Phase Progressive Training (TPT) curriculum that gradually adapts observation and action spaces, improving robustness. These modules form a flexible, interpretable pipeline that enables zero-shot transfer, reducing reliance on learned domain adaptation and extensive real-world data.

• �� GOB employs SegFormer segmentation and inverse perspective mapping to produce BEV semantic maps from monocular images, aligning real observations with simulation data.
• �� PAM maps RL outputs—such as steering angle and throttle—to platform-agnostic commands using a bicycle model with PID control, calibrated via lightweight (~30 min) vehicle-specific parameters.
• �� TPT involves two stages: first, training in simulation with privileged BEV; second, fine-tuning with GOB-generated BEV, gradually bridging observation differences.
• �� RDP integrates GOB and PAM modules into a low-latency (20Hz) closed-loop system, ensuring real-time control.
• �� Theoretical analysis (Theorem 1) bounds the total transfer error by three independent error terms, each controllable via module calibration, guaranteeing performance within specified limits.

The experimental setup includes training a VLM-guided RL policy in CARLA using SAC and CLIP-based semantic rewards. The policy is then deployed on a Ford E-Transit in Madison, WI, without additional real-world RL training. Evaluation metrics include positional error, response latency, and safety margins across scenarios like car-following, obstacle avoidance, and stop signs. Ablation studies compare baseline direct transfer with the proposed GOB, PAM, and TPT modules, demonstrating significant error reduction. The system maintains 20Hz control with response times below 50ms, validating the framework’s robustness. Results confirm the theoretical bounds, with transfer errors controlled within 10%, and real-world performance matching simulation benchmarks.