Adapter-Based Few-Shot Continual Learning for Malicious Packet Recognition
Proposed adapter-based few-shot continual learning framework with SSL pretraining and LoRA, boosting malware packet recognition accuracy.
Key Findings
Methodology
This approach employs domain-specific self-supervised pretraining of a Transformer backbone using Masked Language Modeling (MLM) to learn robust malware packet features. Building on this, LoRA (Low-Rank Adaptation) enables parameter-efficient fine-tuning by injecting low-rank matrices into attention layers, freezing the core backbone. During incremental sessions, a prototype-based classifier constructs class centers by averaging normalized support features, avoiding storage of sensitive data. The entire framework combines SSL, LoRA, and prototype classification, effectively mitigating catastrophic forgetting and enabling secure, scalable incremental learning in dynamic threat environments.
Key Results
- On CIC-IDS2017 and UNSW-NB15 datasets, our method achieves 61.05% and 58.59% accuracy respectively in the final incremental session, outperforming prior rehearsal-free baselines by over 8%. The average forgetting rate drops to 8.59%, half of traditional approaches, demonstrating high stability. Extensive ablation confirms that SSL pretraining combined with LoRA significantly enhances generalization, especially in high-similarity class scenarios.
- Compared to rehearsal-based variants, our rehearsal-free model maintains comparable accuracy while reducing forgetting by more than 50%, highlighting the security and stability advantages of static feature spaces. Ablation studies show SSL and LoRA jointly contribute to robustness, making the model suitable for real-world deployment.
- The model performs well in fine-grained attack subtype recognition, validating its adaptability to complex, evolving malware threats, with potential for real-time security applications.
Significance
This work addresses critical privacy and security constraints in malware detection, proposing a rehearsal-free, SSL-driven, parameter-efficient incremental learning framework. It overcomes the limitations of traditional methods that rely on data storage and full fine-tuning, providing a scalable solution for real-world cybersecurity. The approach enhances model stability, reduces privacy risks, and adapts rapidly to new threats, aligning with industry needs for secure, autonomous defense systems. Its success paves the way for deploying intelligent, privacy-preserving intrusion detection in sensitive environments.
Technical Contribution
This is the first integration of self-supervised pretraining with LoRA-based parameter-efficient fine-tuning for malware FSCIL. The framework introduces a static, robust feature extractor that, once trained, remains fixed during incremental learning, preventing catastrophic forgetting. The prototype classifier enables rapid adaptation without data replay, significantly reducing privacy concerns. Empirical results demonstrate superior performance over state-of-the-art methods, establishing a new paradigm for secure, scalable continual learning in cybersecurity.
Novelty
This work uniquely combines domain-specific SSL pretraining with LoRA adapters, creating a rehearsal-free incremental learning system tailored for malware detection. Unlike existing approaches that depend on large labeled datasets or replay buffers, our method leverages a fixed, pre-trained feature space and lightweight adaptation, addressing privacy, efficiency, and stability simultaneously. It is the first to demonstrate such a synergy in the security domain, marking a significant innovation.
Limitations
- Pretraining requires substantial computational resources and large unlabeled malware datasets, which may limit scalability in resource-constrained environments.
- Frozen feature extractor might limit plasticity, especially when encountering highly divergent or novel attack types outside the pretraining distribution.
- The approach primarily targets Transformer architectures; extending to other models or multimodal data remains future work.
Future Work
Future research will explore dynamic adaptation strategies, such as selectively fine-tuning parts of the backbone, and multi-modal pretraining to enhance robustness. Incorporating continual learning techniques that dynamically adjust model capacity could further improve plasticity. Additionally, deploying this framework in real-world, large-scale security systems will validate its practical effectiveness and scalability.
AI Executive Summary
In the rapidly evolving landscape of cybersecurity, malware variants emerge at an unprecedented pace, challenging traditional detection systems. Conventional models often rely on extensive labeled datasets and retraining, which are impractical in real-time scenarios and pose privacy risks. To address these issues, this study introduces a novel framework that combines self-supervised learning (SSL), parameter-efficient adaptation via LoRA, and prototype-based classification to enable secure, effective incremental malware detection.
The core innovation lies in pretraining a Transformer backbone on unlabeled malware packets using MLM objectives, capturing domain-specific semantic features without requiring labels. During incremental sessions, the model remains frozen, with only lightweight LoRA adapters adjusted, allowing rapid adaptation to new malware classes from scarce samples. The prototype classifier constructs class centers by averaging normalized support features, avoiding the need to store raw data, thus ensuring privacy compliance.
Extensive experiments on CIC-IDS2017 and UNSW-NB15 datasets demonstrate that our approach outperforms existing rehearsal-free methods, achieving over 8% higher accuracy in the final incremental task and significantly reducing forgetting rates. The results validate the effectiveness of combining SSL pretraining with LoRA for stable, scalable, and privacy-preserving malware detection.
This framework offers a promising direction for deploying adaptive intrusion detection systems that can keep pace with evolving threats without compromising data security. Its scalability and robustness make it suitable for real-world cybersecurity applications, paving the way for intelligent, autonomous defense mechanisms that are both secure and efficient.
Deep Analysis
Background
Deep learning在网络安全中的应用不断深化，尤其在恶意流量检测方面取得显著进展。代表性工作如Deep Packet和Packet Inspection Transformer，利用深度模型提取复杂特征，实现高准确率。然而，传统模型在面对新型威胁时，依赖大量标注数据和样本存储，存在隐私泄露和模型灾难性遗忘的问题。近年来，持续学习（Continual Learning）和少样本学习（Few-Shot Learning）成为研究热点，旨在实现模型的动态适应。Transformer架构因其强大表达能力被引入安全领域，结合自监督学习（SSL）技术，减少对标注数据的依赖，提升泛化能力。尽管如此，灾难性遗忘仍是制约模型持续适应的关键难题，尤其在隐私敏感环境中，存储历史样本存在巨大风险。
Core Problem
在恶意软件检测中，模型需要不断适应新出现的攻击类型，传统方法依赖大量标注样本和重放机制，存在隐私泄露和存储风险。此外，模型在多轮增量学习中容易遗忘先前学习的类别，影响整体检测效果。如何在保证数据隐私的前提下，实现模型在少样本条件下的持续适应，成为亟待解决的难题。现有技术多依赖全参数微调或存储历史样本，成本高且不安全，限制了其在实际部署中的应用。
Innovation
本研究提出结合自监督预训练Transformer和LoRA适配器的无重放持续学习框架，创新点包括：1）在大规模无标签恶意包数据上进行MLM预训练，学习鲁棒域特征；2）冻结预训练模型，仅微调轻量级适配器，提升参数效率；3）采用原型分类器，避免存储敏感样本，确保隐私安全。该方案显著降低了灾难性遗忘，提高模型稳定性，同时实现快速适应新类别，满足实际安全场景的需求。与传统全参数微调和重放方法相比，具有更高的安全性和效率。
Methodology
- �� 预训练阶段：利用大量未标注的恶意包数据，采用MLM目标训练Transformer，学习通用语义特征。输入为字节序列，利用掩码预测原始字节，优化交叉熵损失。
- �� 基础会话适配：在预训练基础上，冻结Transformer参数，仅调整LoRA适配器参数，实现对基础类别的高效微调。
- �� 增量学习：新类别支持样本少（K-shot），通过支持样本平均构建类别原型，避免存储敏感数据。原型通过L2归一化后，采用缩放余弦相似度进行分类。
- �� 无重放机制：不存储历史样本，模型参数保持静态，利用预训练特征空间的鲁棒性，持续识别新类别。
- �� 评估指标：准确率和平均遗忘率，验证模型在多轮增量任务中的性能和稳定性。
Experiments
使用CIC-IDS2017和UNSW-NB15两个公开恶意包数据集，按照标准的5-shot增量协议划分训练集和测试集。基础会话（S0）使用500样本/类，后续会话（S1-S3）每类仅支持5个样本。模型在每轮增量后，评估累计类别的准确率和遗忘率，比较重放与非重放方案。超参数包括LoRA秩为8，学习率1e-4，训练三次随机种子取平均。通过消融实验验证SSL预训练和LoRA的贡献。
Results
在两个数据集上，提出方法在第3轮增量任务中，准确率分别达到61.05%和58.59%，优于无重放基线8%以上。遗忘率显著降低至8.59%，比传统方法低一半。模型在不同攻击子类识别中表现出较强适应性，验证了其在复杂威胁场景中的实用性。参数微调结合SSL预训练，有效平衡模型的稳定性和塑性，满足安全环境的需求。
Applications
该模型适用于企业网络安全系统，能在隐私敏感环境中实时识别新型威胁，无需存储敏感样本。可部署于入侵检测系统（IDS）和安全信息事件管理（SIEM）平台，提升动态威胁响应能力。未来可结合多源数据，增强模型泛化能力，支持更大规模的威胁演变监测。
Limitations & Outlook
模型预训练成本较高，依赖大量无标签数据，且在极端域偏移下可能表现不足。冻结特征空间限制了模型的塑性，面对新颖攻击可能识别困难。未来需探索动态调整适配器参数和多模态预训练策略，以提升适应性和鲁棒性。
Plain Language Accessible to non-experts
想象你在一个工厂里工作，工厂每天都在生产不同的商品。为了保证生产效率，你需要一个非常聪明的机器人，它能快速学习新商品的特点，同时记住以前的商品。传统的方法就像让机器人不停地重看所有商品的说明书，既费时间又容易忘记以前学过的内容。而这项研究就像给机器人装上了一个特别的记忆芯片，让它只用少量新信息就能学会新商品，而且还能记住以前的商品。这个记忆芯片是通过让机器人在没有标签的情况下自己学习（自监督学习），然后用一种叫LoRA的轻量级调节工具，只调整少量参数，就能让机器人变得更聪明。这样，工厂的机器人既能快速适应新商品，又不会忘记旧的，既安全又高效。
ELI14 Explained like you're 14
想象你在学校里学新东西，老师给你一些新题目，但只告诉你几次，不能让你记住所有细节。以前的学习方法就像反复看所有的题目，既浪费时间又容易忘记。而这次，老师给你一种特别的学习方法，让你用少量的时间就能理解新题，同时还能记住以前的知识。这就像你有一个超级聪明的脑袋，能自己在没有老师帮忙的情况下，学会新东西，还能记住以前的内容。这个方法用到了一个叫自监督学习的技巧，让电脑自己找规律；用LoRA这个工具，只调整很少的参数，就能学会新东西。这样，电脑就变得又快又稳，不会轻易忘记以前学过的东西，也能快速适应新挑战。
Glossary
Self-Supervised Learning (自监督学习)
一种无需标注数据，通过设计预任务让模型自己学习特征的方法。技术上通过掩码预测等机制实现特征提取，在论文中用于预训练Transformer模型以学习恶意包的语义。
LoRA (Low-Rank Adaptation)
一种参数高效的微调技术，通过引入低秩矩阵调整预训练模型的注意力层参数，减少训练参数量，保持模型稳定性，在增量学习中有效缓解灾难性遗忘。
Prototype Classifier (原型分类器)
一种基于类别原型的分类方法，通过支持样本的特征平均值作为类别代表，利用相似度进行分类，避免存储敏感样本，适合无重放环境。
Catastrophic Forgetting (灾难性遗忘)
模型在连续学习过程中，因学习新任务而丧失旧任务知识的现象，尤其在参数微调或无样本存储时尤为严重。
MLM (Masked Language Modeling, 掩码语言模型)
一种自监督预训练目标，通过掩码部分输入，预测掩码位置的原始内容，用于学习上下文语义特征。
Open Questions Unanswered questions from this research
- 1 未来研究中，如何在极端域偏移或新型威胁出现时，保持模型的塑性和鲁棒性仍是挑战。现有方法多依赖静态特征空间，难以应对完全未知的攻击类型。探索动态调整模型参数或引入多模态信息，将是未来的重要方向。
Applications
Immediate Applications
企业入侵检测系统
可部署于企业网络中，实时识别新型威胁，保护敏感信息。无需存储敏感样本，符合隐私法规，提升检测效率。
安全应急响应平台
结合模型的快速适应能力，提升对新出现攻击的响应速度，减少安全事件的影响。
Long-term Vision
智能安全防御体系
未来可实现全自动化的威胁识别与响应，模型持续学习新威胁，保障网络环境的长期安全。
Abstract
The continual evolution of malware variants necessitates detection systems that can adapt to new threats without retraining from scratch. However, continually updating models on new data often leads to catastrophic forgetting, where previously learned knowledge is overwritten. While continual learning has been increasingly explored for malware detection, the specific setting of Few-Shot Class-Incremental Learning (FSCIL), where new malware classes must be learned from only a small number of labeled examples, remains comparatively underexplored. Therefore, this work investigates the FSCIL setting for malware classification. To address the stability-plasticity dilemma, we propose a hybrid framework that leverages a Self-Supervised Learning (SSL) backbone initialized through domain-specific pre-training on malware packets. Our method incorporates Low-Rank Adaptation (LoRA) to efficiently adapt the model during the base session while freezing the core backbone to preserve previously learned representations, alongside a prototype-based classification head for incremental sessions to establish robust decision boundaries from limited samples. Extensive experiments across several datasets demonstrate that our approach consistently outperforms prior malware FSCIL baselines and achieves state-of-the-art performance.