Precision and Recall for Time Series

Time series anomaly detection has evolved from simple point-based methods like ARIMA and basic thresholding to sophisticated deep learning models such as LSTM-AD and autoencoders. Prior evaluation metrics, notably precision and recall, originated from information retrieval, focusing on individual point correctness. Recent efforts like Numenta NAB introduced early detection biases but remained limited to fixed scoring schemes. As real-world applications demand detection over ranges with partial overlaps and positional importance, existing metrics fall short, prompting the need for more nuanced evaluation frameworks that can accommodate the complexity of range-based anomalies.

Current evaluation metrics inadequately reflect the true performance of range-based anomaly detection algorithms. They ignore partial overlaps, position biases, and multiple overlapping ranges, leading to misleading assessments. This gap hampers the development and comparison of advanced detection methods, especially in critical applications like healthcare and industrial safety, where early and accurate detection over intervals is vital. The challenge lies in designing metrics that are expressive, flexible, and computationally feasible, capable of capturing the nuanced nature of range anomalies while supporting domain-specific preferences.

The paper introduces a range-aware precision and recall framework that extends classical metrics by incorporating overlap size, position bias, and multiple overlaps through tunable functions γ, ω, δ. The model supports adjustable parameters (α) to prioritize existence, size, position, or cardinality, enabling domain-specific customization. This approach maintains compatibility with traditional metrics but significantly enhances their expressiveness, allowing for a more accurate reflection of detection performance in complex scenarios. The theoretical formulation is complemented by empirical validation, demonstrating improved alignment with real detection outcomes.

• �� Define sets of real anomaly ranges R and predicted ranges P. • For each real range Ri, compute recall by evaluating existence, size, position, and cardinality, weighted by α. • Use functions γ, ω, δ to quantify overlap cardinality, size, and positional bias, respectively. • For each predicted range Pi, compute precision similarly, emphasizing overlap quality and bias functions. • Aggregate scores across all ranges, normalizing by total counts, to obtain overall metrics. • Support parameter tuning for application-specific preferences, enabling flexible evaluation. • Implement algorithms that efficiently compute overlaps, apply bias functions, and handle multiple overlaps, ensuring scalability for large datasets.

The evaluation employed datasets like NAB, NYC-Taxi, ECG, and synthetic data, with anomaly ground truths established. Detectors such as LSTM-AD, Greenhouse, and Luminol generated predictions. Parameters α, γ, ω, δ were varied to test sensitivity and customization. Metrics compared included classical point-based precision/recall, the proposed range-based metrics, and NAB scores. Results showed the range-based metrics consistently outperformed classical ones, with recall reaching 0.85 and F1 scores improving by 12%. Parameter adjustments demonstrated the model’s adaptability to early detection, false positive reduction, and positional biases. Cross-dataset analysis confirmed robustness and practical relevance.

The range-aware metrics achieved higher correlation with ground truth, with recall up to 0.85 and F1-score improvements of 12%. They effectively captured partial overlaps and positional biases, unlike classical metrics. Parameter tuning allowed customization for early detection or false alarm minimization, showing flexibility. Comparisons with NAB indicated that the new model could replicate NAB’s early detection bias while providing broader applicability. The experiments validated that the proposed metrics better reflect real detection performance, especially in complex range scenarios, enabling more accurate assessment and algorithm development.

This framework is suitable for industrial equipment monitoring, medical diagnostics, financial fraud detection, and cybersecurity. It provides a comprehensive evaluation tool for developing more sensitive and specific algorithms, supporting real-time decision-making. The customizable parameters allow practitioners to tailor metrics to their operational priorities, such as early warning or false alarm control. Integrating this model into existing detection systems can improve performance assessment, guide algorithm tuning, and foster innovation in anomaly detection methodologies, ultimately enhancing safety and efficiency across sectors.

The model’s reliance on parameter tuning requires domain expertise, which may limit its ease of use. Computational complexity could pose challenges for real-time large-scale streaming data. Overfitting parameters to specific scenarios might reduce generalization. Future work should focus on automating parameter selection, reducing computational load, and extending applicability to multi-modal and multi-source data environments.