EMA: Auditing Data Removal from Trained Models

The rapid adoption of deep learning in sensitive sectors has heightened concerns over data privacy, especially regarding whether models memorize training data. Early research by Shokri et al. introduced membership inference attacks, exposing models’ potential to leak training data. Subsequent work focused on data removal techniques, like Guo et al.'s certified data removal, but lacked robust auditing tools. Existing auditing methods predominantly rely on distribution distance metrics such as KS distance, which are sensitive to data similarity and calibration quality. With regulations like GDPR and HIPAA enforcing strict data privacy, there is an urgent need for reliable, efficient auditing tools capable of verifying data deletion in black-box models, especially in high-stakes fields like medical imaging, where privacy breaches can have severe consequences.

The core challenge is to accurately determine whether a model has memorized or forgotten specific data points under black-box access conditions. Traditional methods relying on KS distance are vulnerable to false negatives when query data resembles training data, or false positives when calibration data quality is poor. These limitations hinder practical deployment, especially in healthcare, where verifying data deletion is crucial for compliance. The difficulty lies in designing an approach that is both computationally efficient and robust against data similarity, calibration noise, and model variations, ensuring reliable privacy auditing without retraining or parameter access.

The main innovations include: 1) integrating multiple membership inference metrics—correctness, confidence, and negative entropy—to capture diverse aspects of model memorization; 2) employing statistical hypothesis testing (two-sample t-test) to aggregate sample-level inferences into a global score, enhancing robustness; 3) eliminating the need for retraining on query data, reducing computational costs. Unlike existing KS-based methods, EMA’s ensemble approach leverages the complementary strengths of multiple metrics and statistical tests, providing a more resilient and adaptable framework for data removal verification. This design addresses the core limitations of prior techniques, especially under challenging conditions such as low-quality calibration data and high data similarity scenarios.

• �� Compute multiple membership inference metrics (accuracy, confidence, entropy) for each query sample using the model outputs.
• �� Train thresholds for each metric on a calibration dataset, optimizing balanced accuracy to distinguish training from non-training data.
• �� For each query sample, compare its metric scores against thresholds to infer membership status.
• �� Collect sample-wise inferences into a vector; perform a two-sample statistical test (e.g., t-test) comparing this vector to an all-one vector, testing the hypothesis that the query set is memorized.
• �� The resulting p-value indicates the likelihood of the query data being retained, serving as the final auditing score.
• �� The process requires only model outputs and a calibration set, avoiding retraining or access to model parameters, making it computationally efficient and scalable.

Experiments utilized MNIST, SVHN, COVIDx, and Childx datasets, with training, calibration, and query sets designed to simulate various data similarity and quality conditions. Models included MLP and ResNet-18 architectures, trained with SGD and Adam optimizers. Hyperparameters such as thresholds for metrics and significance levels for statistical tests were optimized on calibration data. The evaluation focused on detection accuracy, false positive/negative rates, and robustness under calibration noise and data similarity. Comparative analysis with Liu et al.'s KS-based method demonstrated EMA’s superior stability and lower misclassification rates, especially in low-quality calibration scenarios.

EMA achieved over 95% accuracy in detecting memorized training data across datasets, with false positive rates below 5%. In medical imaging, EMA maintained detection accuracy above 92% even when calibration data was noisy (k<60). It outperformed KS-distance methods, which suffered from false positives and negatives under similar conditions. Ablation studies confirmed that combining multiple metrics and statistical tests significantly improved robustness. The results validate EMA’s effectiveness in practical, complex scenarios, including high data similarity and low calibration quality.

EMA can be directly applied for privacy compliance audits in healthcare, finance, and AI services, verifying whether models have forgotten or memorized sensitive data. It supports regulatory requirements like GDPR, enabling organizations to demonstrate data deletion. Additionally, EMA’s efficiency makes it suitable for real-time monitoring in deployed AI systems, ensuring ongoing compliance. Future applications include federated learning environments, where privacy verification is crucial for distributed models, and in continuous learning systems, to prevent data leakage over time.

The approach depends on calibration dataset representativeness; significant distribution shifts can impair accuracy. Extremely noisy calibration sets (k<50) reduce statistical test power, increasing error rates. Compatibility with diverse model architectures and training paradigms remains to be extensively validated. Future work should address adaptive thresholding, multi-modal data, and dynamic environments to improve robustness and generalizability.