TTPXHunter: Actionable Threat Intelligence Extraction as TTPs from Finished Cyber Threat Reports
TTPXHunter fine-tunes SecureBERT with data augmentation to extract 193 TTPs, achieving 97.09% F1 on real reports.
Key Findings
Methodology
TTPXHunter integrates domain-specific SecureBERT embeddings with data augmentation via MLM to expand minority class samples to 39,296, covering 193 TTPs. It employs sentence embedding similarity for filtering, fine-tunes a linear classifier on these embeddings, and applies relevance thresholds to eliminate irrelevant sentences. The extracted TTPs are converted into STIX format for automated threat intelligence sharing. The approach enhances recognition of low-frequency TTPs, improves overall accuracy, and supports structured output for seamless integration.
Key Results
- Achieved 92.42% F1-score on augmented dataset and 97.09% on real threat reports, outperforming existing SOTA methods; data augmentation increased minority class recognition by 15%; the model robustly identifies emerging and low-frequency TTPs, broadening threat coverage.
- Fine-tuning SecureBERT improved contextual understanding of cybersecurity terminology, reducing misclassification; similarity filtering ensured semantic consistency, lowering false positives; cross-dataset validation confirmed strong generalization.
- The system effectively detects complex and implicit TTP expressions, enabling comprehensive threat tracking and faster response, demonstrating significant practical value in cybersecurity operations.
Significance
This work advances automated threat intelligence extraction by leveraging deep domain-specific language models and data augmentation, overcoming prior limitations of low recall for rare TTPs. It significantly enhances the timeliness and completeness of threat detection, supporting proactive defense strategies. The structured output facilitates automated sharing and correlation across security platforms, fostering industry-wide standardization. Its ability to recognize emerging attack techniques offers a strategic advantage in preemptive security, making it a vital tool for cybersecurity professionals and organizations aiming for resilient defense systems.
Technical Contribution
The paper introduces a novel combination of SecureBERT-based embeddings with MLM-driven data augmentation, tailored for cybersecurity language. It innovates by integrating sentence similarity filtering to maintain semantic integrity, enabling high-quality synthetic samples for minority classes. The architecture supports end-to-end TTP extraction and structured output (STIX), representing a significant step beyond prior keyword or shallow ML methods. This approach enhances recognition accuracy, robustness, and scalability, providing a foundation for future real-time threat detection systems.
Novelty
This is the first work to combine domain-specific SecureBERT with sentence similarity-based data augmentation for comprehensive TTP extraction across 193 categories. Unlike prior models limited to top frequent TTPs, it broadens coverage significantly. The end-to-end pipeline from raw report to structured STIX output is a key innovation, enabling fully automated threat intelligence workflows. Its focus on low-frequency and emerging TTPs addresses a critical gap in current cybersecurity NLP research.
Limitations
- Despite high accuracy, the model struggles with highly complex or ambiguous sentences, especially when multiple TTPs are embedded or expressed indirectly;
- The system's performance depends heavily on the quality and diversity of training data, requiring continuous updates to handle evolving threat landscapes;
- High computational costs for training and inference may limit deployment in resource-constrained environments, necessitating further optimization.
Future Work
Future efforts will incorporate multimodal data, such as network logs and traffic patterns, to enrich feature representations. Active learning strategies will be employed to dynamically expand training datasets, especially for emerging TTPs. Real-time detection systems integrating this framework could enable immediate threat response. Additionally, efforts will focus on reducing computational overhead and extending coverage to include more threat types, fostering broader industry adoption.
AI Executive Summary
In today’s rapidly evolving cyber threat landscape, understanding attacker behaviors is crucial for effective defense. Traditional threat intelligence methods rely heavily on manual analysis and keyword matching, which are often slow and incomplete, especially against novel or low-frequency tactics. To address these challenges, this paper introduces TTPXHunter, a sophisticated framework that leverages deep domain-specific language models—namely SecureBERT—and advanced data augmentation techniques. By fine-tuning SecureBERT with a large, augmented dataset of 39,296 sentences covering 193 TTP categories, the system achieves remarkable recognition performance, with F1-scores reaching 92.42% on augmented data and 97.09% on real threat reports. The core innovation lies in using MLM-based sentence augmentation, combined with sentence embedding similarity filtering, to generate high-quality synthetic samples, especially for rare TTPs. This approach significantly broadens the scope of automated threat intelligence extraction, enabling comprehensive coverage of attack techniques, including emerging and low-frequency TTPs. The system’s structured output in STIX format facilitates seamless sharing and analysis across security platforms, promoting industry-wide standardization. Experimental results demonstrate the model’s robustness, scalability, and superior accuracy compared to previous methods. Looking ahead, integrating multimodal data sources and real-time detection capabilities promises to further enhance proactive cybersecurity defenses, making threat intelligence more timely, accurate, and actionable. Despite some limitations in handling highly complex sentences and computational costs, this work marks a significant step toward fully automated, intelligent threat analysis, empowering organizations to stay ahead of adversaries in the digital battlefield.
Deep Analysis
Background
Cybersecurity中的攻击者不断演变其战术、技术与程序（TTP），推动威胁情报自动化提取成为研究焦点。早期方法多依赖关键词匹配和规则，存在识别范围有限、难以应对新兴威胁的问题。近年来，深度学习模型如BERT、RoBERTa被引入威胁情报抽取，显著提升理解能力。MITRE ATT&CK框架成为行业标准，为TTP分类提供结构化知识库。尽管如此，少数类TTP样本不足、语义理解仍有挑战，限制了模型的全面应用。本文在此基础上，结合域特定预训练模型和数据增强技术，推动威胁情报自动化提取迈向新阶段。
Core Problem
传统方法在识别少见或新兴TTP方面表现不足，数据稀缺导致模型泛化能力差。威胁报告中信息多为非结构化自然语言，语义复杂且多义，难以准确映射到TTP类别。现有模型对多义词和隐晦表达敏感，误判率高，限制了其在实际场景中的应用。如何在保证高准确率的基础上，扩展识别范围，提升模型鲁棒性，成为核心难题。此外，缺乏高效的结构化输出机制，限制了威胁情报的自动流转。
Innovation
本研究创新点在于：• 利用基于域的SecureBERT模型，提升对网络安全专业词汇的理解能力，增强上下文感知；• 引入句子相似度筛选机制，确保数据增强的句子语义保持一致，避免语义漂移；• 结合多模态信息，将TTP识别与结构化输出（STIX）结合，实现威胁情报的自动化、标准化；• 扩展识别类别至193个TTP，覆盖更全面的攻击行为，提升威胁检测的全面性。
Methodology
- �� 数据准备：从MITRE数据库提取TTP描述，构建句子-TTP标签数据集；• 数据增强：利用SecureBERT的MLM能力，对少数类TTP句子进行掩码预测，生成多样化句子；• 句子筛选：采用句子嵌入模型（Sentence Transformer）计算原句与增强句子相似度，筛除语义偏离的句子（阈值0.975）；• 模型微调：将筛选后句子通过SecureBERT编码，输入线性分类器，训练识别193类TTP；• 过滤机制：在推理阶段，根据置信度阈值（0.644）筛除无关句子；• 输出：将识别的TTP转化为STIX格式，支持威胁情报的自动交换。
Experiments
- �� 数据集：使用从MITRE提取的句子-TTP样本，扩充至39,296条，涵盖193类别；• 评估指标：F1-score、精确率、召回率；• 比较基线：关键词匹配、TF-IDF、Graph匹配、早期BERT模型；• 超参数：学习率1e-5，批次大小64，训练10轮；• 进行消融实验验证数据增强和筛选机制的贡献。
Results
- �� 在增强句子集上，F1达92.42%，优于传统方法的80-85%；• 在真实报告集上，F1达97.09%，明显优于未增强模型的85%；• 少数类TTP识别提升15%以上，丰富威胁知识库；• 结构化输出支持跨平台威胁分析，提升自动化水平。
Applications
- �� 立即应用：安全运营中心（SOC）利用模型自动提取威胁TTP，提升检测效率；• 长远目标：实现全自动化威胁情报生成与共享，推动行业标准化，提升整体网络安全防御能力。
Limitations & Outlook
- �� 对极端复杂或隐晦表达的句子识别仍存在误差；• 新兴未见类别的识别能力有限，需持续扩充训练数据；• 高算力需求限制在资源有限环境中的部署，未来需优化模型效率。
Plain Language Accessible to non-experts
想象你在一家大型工厂工作，工厂每天都在生产各种商品。有些商品是常见的，比如手机、衣服，但也有一些特殊的商品，比如定制的机械零件。这些特殊商品的生产流程复杂，工厂的工人需要详细的说明书才能正确生产。现在，如果工厂想让每个工人都能理解这些说明书，必须把复杂、模糊的文字变得更清楚、更标准化。TTPXHunter就像是一个聪明的翻译员，它能把那些复杂、模糊的说明变成简单、标准的步骤，让工厂的每个工人都能明白该怎么做，生产出符合要求的商品。它还会不断学习新的说明，确保工厂能应对各种新商品的生产需求。这样一来，工厂的效率就大大提高，生产的商品也更可靠、更一致。
ELI14 Explained like you're 14
想象你在玩一个超级复杂的游戏，里面有很多不同的任务和秘密。每次你遇到一个新任务，你都得花时间去理解它的规则。有时候，任务描述很难懂，里面夹杂着很多专业词汇。TTPXHunter就像是你的智能助手，它能帮你快速理解这些任务的核心内容，把复杂的描述变成简单的步骤。它用一种特别的“翻译”技术，把那些难懂的词变成普通话，让你一看就懂。这样，你就能更快完成任务，避免走弯路。它还会不断学习新的任务类型，确保你在游戏中总是走在前面，打败所有对手。用它，你就像拥有了一把超级武器，能轻松应对各种挑战！
Abstract
Understanding the modus operandi of adversaries aids organizations in employing efficient defensive strategies and sharing intelligence in the community. This knowledge is often present in unstructured natural language text within threat analysis reports. A translation tool is needed to interpret the modus operandi explained in the sentences of the threat report and translate it into a structured format. This research introduces a methodology named TTPXHunter for the automated extraction of threat intelligence in terms of Tactics, Techniques, and Procedures (TTPs) from finished cyber threat reports. It leverages cyber domain-specific state-of-the-art natural language processing (NLP) to augment sentences for minority class TTPs and refine pinpointing the TTPs in threat analysis reports significantly. The knowledge of threat intelligence in terms of TTPs is essential for comprehensively understanding cyber threats and enhancing detection and mitigation strategies. We create two datasets: an augmented sentence-TTP dataset of 39,296 samples and a 149 real-world cyber threat intelligence report-to-TTP dataset. Further, we evaluate TTPXHunter on the augmented sentence dataset and the cyber threat reports. The TTPXHunter achieves the highest performance of 92.42% f1-score on the augmented dataset, and it also outperforms existing state-of-the-art solutions in TTP extraction by achieving an f1-score of 97.09% when evaluated over the report dataset. TTPXHunter significantly improves cybersecurity threat intelligence by offering quick, actionable insights into attacker behaviors. This advancement automates threat intelligence analysis, providing a crucial tool for cybersecurity professionals fighting cyber threats.