SafeTune: Mitigating Data Poisoning in LLM Fine-Tuning for RTL Code Generation
SafeTune combines GNN and semantic analysis to filter poisoned RTL data, enhancing LLM fine-tuning security.
Key Findings
Methodology
SafeTune integrates a GNN-based structural anomaly detector with GTE-large text embeddings and XGBoost risk scoring. It converts RTL designs into Data-Flow Graphs (DFGs), trains GNNs to identify Trojan-like structures, and encodes prompt texts into embeddings for semantic risk assessment. The framework filters out high-risk samples before fine-tuning, effectively reducing Trojan learning. This approach requires no modification to the base models, maintaining performance while enhancing robustness.
Key Results
- On VerilogEval, SafeTune fine-tuned models show a drastic reduction in attack success rate (ASR), with Qwen2.5-Coder-14B dropping from 94% to 33%, and CodeLlama-13B from 96% to 37%. RTL functional correctness remains stable, with Pass@k scores comparable to baseline models.
- Combining offline dataset filtering with runtime prompt paraphrasing further decreases ASR, achieving 33-37% overall success, demonstrating multi-layer defense effectiveness.
- Across multiple Trojan types, the combined method significantly mitigates backdoor activation without degrading RTL quality, validating its practical utility.
Significance
This work addresses a critical gap in hardware design security, providing a scalable, model-agnostic framework that jointly analyzes structure and semantics to detect and filter poisoned data. It offers industry-ready solutions to prevent hardware Trojans during automated RTL generation, thereby strengthening the trustworthiness of AI-assisted hardware design workflows.
Technical Contribution
The paper introduces a novel multi-modal filtering framework that combines GNN structural analysis with semantic risk scoring via XGBoost. It innovates by applying graph-based Trojan detection to RTL data and embedding-based prompt evaluation, enabling effective pre-training data sanitization without model architecture changes. The integration of offline filtering and runtime paraphrasing forms a comprehensive defense system.
Novelty
This is the first work to jointly utilize GNNs and large-scale text embeddings for RTL Trojan detection, addressing both structural anomalies and semantic triggers in a unified cross-modal framework. It advances beyond prior methods that focus solely on either structure or semantics, offering a holistic security solution.
Limitations
- The method's effectiveness depends on the quality of pre-trained models; extremely sophisticated Trojans may evade detection. Computational costs for large-scale graph processing remain high, limiting real-time deployment in massive designs.
- Current approach primarily targets static poisoning; adaptive or dynamic attacks might require further enhancements. Future work should incorporate online learning and dynamic validation mechanisms.
Future Work
Future directions include developing more efficient graph analysis algorithms, integrating dynamic runtime validation, and extending the framework to other hardware description languages. Exploring adaptive defense strategies against evolving attack vectors will further strengthen security.
AI Executive Summary
The rapid adoption of large language models (LLMs) in hardware design, especially for RTL code generation, has revolutionized automation but also introduced significant security vulnerabilities. Unverified datasets, often assembled hastily or generated automatically, are susceptible to poisoning attacks that embed hardware Trojans—malicious logic that can activate under specific conditions, threatening hardware integrity.
Traditional defenses focus on post-silicon detection or functional correctness validation, which are insufficient against stealthy backdoors embedded during training. Recognizing this gap, SafeTune proposes a comprehensive, multi-modal filtering framework that jointly analyzes RTL structural features and prompt semantics. The core innovation lies in leveraging a graph neural network (GNN) to detect Trojan-like structures within RTL Data-Flow Graphs, combined with a semantic risk scoring system based on large-scale text embeddings and XGBoost classifiers. This dual analysis enables preemptive filtering of poisoned data before fine-tuning, significantly reducing the likelihood of Trojan learning.
Experimental results demonstrate the effectiveness of SafeTune across multiple models and Trojan types. For instance, the attack success rate (ASR) on Qwen2.5-Coder-14B drops from 94% to 33%, while RTL functionality remains intact. When combined with runtime prompt paraphrasing, the overall security improves further, creating a robust defense-in-depth system. These findings highlight a practical, model-agnostic approach that enhances the security of AI-assisted hardware design workflows.
The impact extends beyond immediate security gains; it establishes a scalable paradigm for integrating structural and semantic analysis into hardware design pipelines. Limitations include computational costs and challenges against highly sophisticated or adaptive attacks. Future work aims to optimize algorithms, incorporate dynamic validation, and broaden applicability to diverse hardware description languages, paving the way for safer, more trustworthy hardware automation.
Deep Dive
Abstract
As large language models (LLMs) are increasingly fine-tuned for hardware tasks like RTL code generation, the scarcity of high-quality datasets often leads to the use of rapidly assembled or generated training data. These datasets frequently lack security verification and are highly susceptible to data poisoning attacks. Such poisoning can cause models to generate syntactically valid but insecure hardware modules that bypass standard functionality checks. To address this, we present SafeTune, a framework designed to harden LLM-based RTL generation against poisoning, specifically focusing on hardware Trojan (HT) insertion. SafeTune integrates two core components: (i) a Graph Neural Network (GNN) that models structural properties to identify anomalous circuitry patterns during fine-tuning, and (ii) a semantic verification module using text embeddings and an XGBoost classifier to assess prompt security. By coupling structural and semantic knowledge, SafeTune effectively filters poisoned inputs without sacrificing legitimate data. Experimental results demonstrate that SafeTune significantly enhances the robustness and reliability of LLM fine-tuning without requiring modifications to the underlying model architecture.