Looking Beyond IoCs: Automatically Extracting Attack Patterns from External CTI

Cyber Threat Intelligence (CTI) has evolved from simple indicator-based detection to complex behavioral analysis, leveraging frameworks like MITRE ATT&CK. Early methods focused on static IoCs, which are easy to modify by adversaries, limiting their long-term effectiveness. Recent research emphasizes attack techniques, tactics, and procedures (TTPs), aiming for a deeper understanding of adversary behaviors. However, extracting meaningful attack patterns from vast, unstructured reports remains challenging due to noisy data and linguistic ambiguity. Existing tools like STIX and TAXII facilitate structured sharing but are limited by manual effort and reporting delays. Deep learning models, especially transformers, have shown promise in NLP tasks, yet integrating them with knowledge graphs for attack pattern inference is still nascent. This paper situates itself at the intersection of NLP, knowledge representation, and cybersecurity, proposing a comprehensive solution to automate attack behavior extraction and standardization, thus significantly advancing threat intelligence automation.

Current threat intelligence practices rely heavily on IoCs, which are often transient and insufficient against sophisticated attacks. The core challenge is extracting high-level attack behaviors from unstructured textual reports, which are noisy, diverse, and complex. Manual analysis is labor-intensive and slow, unsuitable for real-time defense needs. Rule-based systems lack flexibility and scalability, especially when facing novel attack descriptions. The difficulty lies in accurately identifying attack patterns, mapping them to standardized frameworks like ATT&CK, and integrating this information into actionable intelligence. Addressing these issues requires developing models capable of understanding natural language at scale, recognizing nuanced attack descriptions, and translating them into structured, machine-readable formats. This paper tackles these challenges by proposing an automated, scalable framework that leverages deep NLP and knowledge graph reasoning to extract, classify, and predict attack behaviors, thereby enhancing the timeliness and depth of threat intelligence.

The key innovations include: 1) Combining transformer-based NLP models with ontology-driven knowledge graphs to automate attack pattern extraction from unstructured CTI reports, enabling large-scale analysis. 2) Developing TTPClassifier, a machine learning model that classifies attack patterns into MITRE ATT&CK techniques with high accuracy, facilitating standardized threat representation. 3) Curating and releasing an open malware dataset with detailed annotations, supporting reproducibility and future research. 4) Integrating knowledge graph reasoning to infer potential unseen attack strategies, providing predictive threat insights. These innovations collectively address the limitations of IoC-centric approaches, enabling a more comprehensive, scalable, and proactive threat intelligence system.

• �� Data collection: Developed a web crawler to scrape over 12,000 CTI reports from public sources, focusing on malware like Cerberus, Rotexy, Judy.
• �� Data annotation: Used BRAT tool for manual labeling of entities (malware, attack patterns, applications, locations) and relationships, creating a high-quality training set.
• �� Entity recognition: Fine-tuned transformer models (BERT, RoBERTa, XLM-R) on annotated data to identify entities in unstructured text, supplemented by regex-based IoC extraction.
• �� Attack pattern extraction: Implemented a three-step process—sentence classification to find relevant descriptions, sequence tagging to identify attack phrases, and mapping to MITRE ATT&CK IDs.
• �� Knowledge graph construction: Transformed entities and relations into RDF triples, aligned with a predefined ontology, and built a comprehensive attack behavior graph.
• �� Inference: Applied graph reasoning to predict potential attack strategies not explicitly reported.
• �� Evaluation: Used metrics like F1-score and accuracy to validate entity recognition and attack pattern classification across multiple datasets and scenarios.

The experiments involved training on 36 annotated malware reports, testing on unseen reports for generalization. Metrics included F1-score (>85%) for entity recognition, 78% accuracy for attack pattern classification, and inference validation through known and novel attack scenarios. Baseline comparisons with rule-based extraction and classical ML models confirmed the superiority of the deep learning approach. Cross-source and cross-attack type evaluations demonstrated robustness. Ablation studies assessed the contribution of each component, such as the transformer models and knowledge graph reasoning, confirming their effectiveness. The system was also tested in real-world threat scenarios, like Cerberus, to evaluate practical utility, showing significant improvements in early threat detection and proactive defense.

LADDER achieved high accuracy in extracting attack behaviors, with over 85% F1-score in entity recognition and 78% in attack pattern classification. It successfully identified multiple attack tactics of malware like Cerberus, enabling early warnings that improved detection efficiency by more than 30%. The knowledge graph's inference capabilities allowed prediction of unseen attack strategies, demonstrating its potential for proactive threat hunting. Cross-source validation confirmed consistent performance, highlighting its scalability. These results underscore the framework's ability to transform unstructured CTI into actionable, predictive intelligence, significantly advancing automated threat analysis.

This framework can be integrated into enterprise security operations centers, automating the analysis of large volumes of CTI reports for rapid threat detection. Security analysts can leverage the knowledge graph for attack path analysis and threat attribution, enhancing incident response. The system supports proactive defense by predicting future attack techniques based on current intelligence. Additionally, it can be used by cybersecurity agencies for national threat monitoring, enabling coordinated and automated threat mitigation strategies. The approach also facilitates threat intelligence sharing across organizations by standardizing attack patterns within a common framework.

Despite its strengths, the system faces challenges such as reduced accuracy when处理复杂句子或新兴攻击技术，模型对训练数据的依赖限制了其泛化能力。知识图谱关系的预定义可能无法涵盖所有新型攻击关系，动态更新机制仍需优化。此外，模型在多语言环境下的表现有限，未来需引入多语种训练和在线学习机制，以适应不断变化的威胁环境。