Real-time Out-of-distribution Detection in Learning-Enabled Cyber-Physical Systems

The evolution of autonomous systems has driven the adoption of learning-enabled components (LECs) such as deep neural networks for perception, planning, and control. Notable prior works include Deep Neural Networks for perception tasks and generative models like Variational Autoencoders (VAE) and Generative Adversarial Networks (GANs) for anomaly detection. Despite these advances, safety concerns persist due to the models' inability to identify inputs outside their training distribution, especially in high-dimensional sensor environments like LIDAR, camera, and radar data. Traditional methods such as k-NN and kernel density estimation face scalability issues in high dimensions. Recently, conformal prediction techniques have emerged, offering statistical guarantees and calibration, but their application in real-time CPS remains limited. This background sets the stage for developing scalable, robust, and calibrated detection methods suitable for safety-critical applications.

The core challenge is to realize real-time, accurate detection of out-of-distribution inputs in high-dimensional, dynamic CPS environments. Existing methods often lack scalability or calibration guarantees, leading to high false alarm rates or missed detections. The difficulty lies in balancing computational efficiency with detection robustness, especially under environmental variability and sensor noise. Moreover, ensuring that the detection system maintains a low false positive rate while being sensitive enough to identify genuine anomalies is critical. These issues are compounded by the need for the system to operate continuously and adaptively, making the problem both technically and practically demanding.

Key innovations include: 1) integrating VAE's generative sampling with SVDD's hypersphere-based anomaly detection to compute nonconformity scores efficiently; 2) employing ICAD to calibrate false alarm rates, ensuring statistical validity; 3) utilizing sliding window and martingale tests to handle temporal correlations in sensor data, enhancing robustness. This approach addresses the limitations of prior distance-based methods by leveraging deep generative models for scalable, high-dimensional data processing. It also introduces a novel calibration mechanism that guarantees error control, making it suitable for safety-critical CPS. The combination of these techniques results in a detection system that is both fast and reliable, with strong theoretical backing.

• �� Model training:
• Use training data to train VAE and deep SVDD models, capturing data distribution features.
• Split data into proper training and calibration sets for nonconformity score calibration.
• �� Online detection:
• For each new input, encode with VAE, sample multiple reconstructions, compute reconstruction errors as nonconformity scores.
• Calculate p-values based on calibration scores, using sorted calibration nonconformity values.
• Apply sliding window and martingale tests to sequence of p-values, detecting anomalies.
• Trigger alarms if statistical tests indicate out-of-distribution inputs.
• �� Statistical guarantees:
• Use ICAD to ensure false alarm rate calibration.
• Incorporate temporal correlation handling via sliding window and martingale-based hypothesis testing.

Experiments conducted in CARLA simulator involve scenarios like adverse weather and road attacks. The models are trained on normal driving data, then tested with out-of-distribution inputs generated by environmental variations and physical attacks. Metrics include false positive rate, detection delay, and accuracy. Baseline comparisons involve traditional k-NN and KDE methods, demonstrating superior scalability and robustness of the proposed approach. Hyperparameters such as sampling number N and detection thresholds are tuned via cross-validation. Multiple runs confirm consistency, and ablation studies evaluate the contribution of each component, validating the effectiveness of the combined VAE, SVDD, and ICAD framework.

Results show that the proposed method maintains false alarm rates below 2%, with detection delays less than 1 second across various scenarios. It accurately detects physical attacks like painted road lines with over 95% accuracy, outperforming traditional methods significantly. The models exhibit stable performance under different weather conditions, sensor noise, and attack types, confirming robustness. The computational overhead remains comparable to original ML inference, validating real-time applicability. These findings demonstrate the method's potential for deployment in safety-critical CPS, providing early warning and reducing risk of failures.

This detection framework can be integrated into autonomous vehicles, industrial control systems, and UAVs to provide real-time safety monitoring. It requires only trained models and calibration data, making it adaptable to various sensor configurations. The approach enhances resilience against environmental variability and malicious attacks, supporting industry standards for safety and reliability. Future integration with edge computing and adaptive control strategies can further improve system robustness and operational efficiency.

Despite promising results, the method's effectiveness depends on the representativeness of training data. Extreme environmental conditions or novel attack types may challenge detection accuracy. Computational costs for sampling and inference, especially in resource-constrained devices, remain a concern. The models assume stationarity of normal data distribution, which may not hold in highly dynamic environments. Further research is needed to improve adaptability, reduce latency, and extend to unsupervised or semi-supervised settings.