ASTER: Latent Pseudo-Anomaly Generation for Unsupervised Time-Series Anomaly Detection
ASTER generates pseudo-anomalies in latent space, combining Transformer and pre-trained LLMs for unsupervised time-series anomaly detection, outperforming existing methods.
Key Findings
Methodology
ASTER employs a Variational Autoencoder (VAE) to learn the normal data distribution in latent space, generating diverse, challenging pseudo-anomalies. Pre-trained LLMs extract rich contextual features from time windows, mapping them into high-dimensional space. A Transformer classifier is trained to distinguish normal from pseudo-anomalous samples. Pseudo anomalies are produced via a learned perturbator, avoiding handcrafted augmentation. The process involves feature extraction, latent perturbation, pseudo anomaly generation, and classification. Experiments on three benchmarks show superior performance, with F1 scores exceeding 0.512 and significant improvements over baselines.
Key Results
- On PSM, PUMP, SWaT, and CATSv2 datasets, ASTER achieves an average F1-score of 0.512, outperforming the best baseline (0.186). AUROC reaches 0.697, AUPR 0.501, indicating strong robustness. Ablation studies confirm the importance of the latent perturbator and Transformer classifier. The model maintains high performance across various anomaly types and window lengths, demonstrating excellent generalization.
Significance
This work advances unsupervised time-series anomaly detection by enabling automatic, diverse pseudo anomaly generation directly in latent space, removing reliance on domain-specific rules. Combining the contextual understanding of large language models with flexible generative mechanisms, ASTER significantly improves detection accuracy and robustness. It addresses long-standing challenges in industrial fault detection, healthcare monitoring, and cybersecurity, offering a scalable, adaptable solution. The approach bridges deep generative modeling with powerful sequence understanding, opening new avenues for multi-modal, multi-task anomaly detection in complex environments.
Technical Contribution
ASTER’s key innovations include: 1) learning a normal data distribution in latent space via VAE, enabling automatic pseudo anomaly synthesis; 2) leveraging pre-trained LLMs for rich contextual feature extraction; 3) employing a Transformer classifier for flexible, non-linear decision boundaries. Unlike traditional reconstruction-based methods, ASTER focuses on latent space perturbation and generation, providing a more versatile anomaly modeling framework. The adversarial-like training of the pseudo-anomaly generator enhances sample difficulty, improving model robustness. The integration of these components results in a novel, highly effective unsupervised detection pipeline.
Novelty
This is the first work to systematically generate pseudo anomalies directly within the latent space using learned distributions, combined with large pre-trained models for feature extraction. Unlike prior methods relying on handcrafted rules or fixed distance metrics, ASTER’s approach is data-driven, adaptive, and domain-agnostic. The fusion of VAE-based generative modeling, LLM contextual encoding, and Transformer classification constitutes a significant innovation, setting a new standard for unsupervised TSAD.
Limitations
- Performance may degrade with extremely rare or novel anomalies not well represented in the latent distribution. The pseudo anomaly generator’s effectiveness depends on the quality of the learned normal distribution.
- Training complexity and computational cost are higher due to the combined VAE, LLM, and Transformer components, limiting real-time deployment in resource-constrained environments.
- Model interpretability remains challenging, especially regarding the nature of generated pseudo anomalies and decision boundaries.
Future Work
Future research will explore multi-modal data fusion, incorporating spatial-temporal and visual information. Enhancing the efficiency and scalability of the model for real-time applications is also a priority. Improving interpretability and explainability of the detection decisions will facilitate industrial adoption. Additionally, extending the framework to semi-supervised or weakly supervised settings could further boost performance in scenarios with limited normal data.
AI Executive Summary
Time-series anomaly detection is vital across many sectors, yet existing methods often struggle with the diversity and scarcity of labeled anomalies. Traditional approaches rely heavily on domain-specific rules, reconstruction, or forecasting models, which falter in complex, real-world scenarios. Recent advances in deep learning and large language models (LLMs) have opened new possibilities, but integrating these effectively remains a challenge.
This paper introduces ASTER, a novel framework that leverages the power of generative modeling in latent space combined with contextual feature extraction via pre-trained LLMs. The core idea is to learn the normal data distribution using a Variational Autoencoder (VAE), then generate diverse pseudo anomalies directly in this space. These synthetic anomalies are used to train a Transformer-based classifier capable of capturing complex, non-linear patterns. The approach avoids handcrafted rules and domain-specific augmentations, making it highly adaptable.
The experimental results on four benchmark datasets demonstrate ASTER’s superiority, with F1 scores surpassing 0.512 and significant improvements over existing methods, including those based on large models. The model’s robustness is validated through multiple metrics, including AUROC, AUPR, and VUS, showing consistent performance across different anomaly types and environments. The integration of latent space generative modeling with contextual encoding marks a significant step forward in unsupervised anomaly detection.
The broader impact of this work lies in its potential to transform industrial monitoring, healthcare, and cybersecurity. By automating anomaly generation and detection without domain-specific tuning, ASTER offers a scalable, generalizable solution. Future directions include multi-modal data fusion, real-time deployment, and enhanced interpretability, aiming to make anomaly detection more accurate, efficient, and explainable in complex real-world systems.
Deep Analysis
Background
时间序列异常检测经历了从传统统计方法到深度学习的演变。早期采用自回归模型、假设检验和距离度量，难以捕获复杂非线性关系。近年来，LSTM、VAE、GAN等深度模型显著提升了检测能力，但仍受限于对异常类型的依赖和泛化能力。Transformer模型引入长距离依赖建模，增强了多变量时间序列的表达能力。尽管如此，如何在无标签条件下自动生成多样化的伪异常，仍是研究难点。近年来，大模型在自然语言处理中的成功激发了其在时间序列中的应用潜力，推动了跨领域的创新。
Core Problem
核心问题在于：如何在无监督环境中有效识别复杂、多样的异常行为？传统方法多依赖预定义异常类型或手工增强，难以应对未知或新颖的异常。现有的嵌入空间方法受限于距离度量的固定性，难以捕获复杂的异常特征。缺乏自动、多样化的伪异常生成机制，限制了模型的泛化能力。这些问题在工业监控、医疗诊断和网络安全中尤为突出，亟需一种无需领域知识、能自动生成多样异常的通用框架。
Innovation
ASTER的创新点在于：1）在潜在空间中学习正常数据分布，通过VAE实现多样化伪异常生成，避免手工设计规则；2）引入预训练大模型（LLM）进行上下文特征编码，增强时间依赖理解；3）利用Transformer分类器实现非线性决策边界，提升复杂异常检测能力。这些创新结合了生成模型、深度特征提取和强大分类能力，突破了传统方法在复杂场景中的局限。
Methodology
- �� 特征提取：利用预训练LLM（如GPT-2）将时间窗口映射到高维空间，获得丰富的上下文信息。• 潜在扰动：通过变分自编码器（VAE）学习正常数据的潜在分布，定义潜在空间的概率模型。• 伪异常生成：在潜在空间中采样，利用扰动器（由VAE的解码器和对抗机制）生成多样化、难以检测的伪异常。• 分类训练：Transformer分类器接收上下文特征，学习区分正常与伪异常。• 训练策略：采用逆梯度优化伪异常生成器，确保生成样本具有挑战性和多样性。• 评估：在多个公开数据集上进行性能测试，比较F1、AUROC、AUPR和VUS指标，验证模型优越性。
Experiments
采用PSM、PUMP、SWaT和CATSv2四个多变量时间序列数据集，涵盖工业、医疗和水处理场景。对比11个基线方法，包括深度学习、预训练模型和LLM方法。指标包括F1、AUROC、AUPR和VUS，评估模型在不同阈值和鲁棒性上的表现。通过消融实验验证潜在扰动器和Transformer分类器的贡献。调优关键超参数如潜在空间维度和扰动强度，确保公平比较。结果显示ASTER在所有指标上均优于对比方法，特别是在F1和VUS指标上表现突出。
Results
ASTER在PSM数据集上的F1-score达到0.512，明显优于最接近的基线（0.186）；在PUMP、SWaT和CATSv2上也有类似提升。AUROC平均值达0.697，AUPR达0.501，显示模型在不同场景下的鲁棒性。消融实验表明，潜在扰动器和Transformer分类器的结合是性能提升的关键。模型在不同异常类型和时间窗口长度下均表现优异，验证了其泛化能力。相比传统重建或预测模型，ASTER在复杂、多变的环境中更具适应性。
Applications
该方法适用于工业设备监控、医疗健康监测和网络安全等场景，能够在无标签条件下自动检测异常。只需提供正常数据样本，无需手工定义异常类型，便可实现高效监测。模型可部署在边缘设备或云端，实时处理大规模数据，提升系统的安全性和可靠性。未来结合多模态信息，将拓展其应用范围，支持更复杂的场景需求。
Limitations & Outlook
模型在极端或罕见异常类型上可能表现不足，因伪异常生成依赖潜在空间表达能力。训练过程中对参数敏感，调优复杂。高维复杂数据和大规模数据集带来较高计算成本，需优化模型结构或采用剪枝策略。未来需增强模型的解释性和实时性，提升实际工业应用的可行性。
Plain Language Accessible to non-experts
想象你在一个工厂里，工厂每天都生产不同的产品。监控系统的任务是检测出任何异常，比如机器突然停工或出现故障。传统的方法就像提前写好一些规则，比如“机器停了就报警”，但这些规则不能应对所有可能的异常。ASTER就像让系统自己学习正常的生产流程，然后用一种“魔法”在内部空间模拟出各种可能的故障情况，这些模拟的故障就像是工厂里故障的“假样本”。这样，系统就能更聪明地识别真正的异常，不管它们长得多奇怪。它还用了一种“翻译员”——预训练的大模型，把时间序列变成丰富的故事，让检测更有“故事感”。最终，这个系统可以在没有提前告诉它所有异常类型的情况下，自动发现工厂里的任何异常，帮助工厂更安全、更高效地运行。
ELI14 Explained like you're 14
想象你在学校，有一台超级厉害的老师，它每天观察你和同学们的表现，想知道谁在学习上出了问题。以前，老师只会用一些简单的规则，比如“成绩突然变差就提醒”，但这些规则不能发现所有隐藏的问题。ASTER就像是让老师自己学会正常的学习模式，然后用一种聪明的“魔法”在脑海里模拟出各种可能的学习问题，比如偷偷作弊或忘记带书的情况。这样，老师就能更早、更准确地发现问题，不管这些问题看起来多奇怪。它还用了一种特别的“故事讲述”技巧，把每天的学习情况变成丰富的故事，让老师理解得更深入。最终，这个系统可以在没有提前告诉它所有问题的情况下，自动找到学习上的异常，帮助学生和老师都变得更聪明、更有准备。
Abstract
Time-series anomaly detection (TSAD) is critical in domains such as industrial monitoring, healthcare, and cybersecurity, but it remains challenging due to rare and heterogeneous anomalies and the scarcity of labelled data. This scarcity makes unsupervised approaches predominant, yet existing methods often rely on reconstruction or forecasting, which struggle with complex data, or on embedding-based approaches that require domain-specific anomaly synthesis and fixed distance metrics. We propose ASTER, a framework that generates pseudo-anomalies directly in the latent space, avoiding handcrafted anomaly injections and the need for domain expertise. A latent-space decoder produces tailored pseudo-anomalies to train a Transformer-based anomaly classifier, while a pre-trained LLM enriches the temporal and contextual representations of this space. Experiments on three benchmark datasets show that ASTER achieves state-of-the-art performance and sets a new standard for LLM-based TSAD.