Towards Compositional Generalization in LLMs for Smart Contract Security: A Case Study on Reentrancy Vulnerabilities
Proposes a post-training atomic task decomposition and fusion algorithm, achieving 98.2% accuracy in reentrancy vulnerability detection with enhanced out-of-distribution generalization.
Key Findings
Methodology
This work employs an atomic task decomposition and fusion strategy, breaking down reentrancy detection into four linearly independent subtasks: external call identification, state update detection, data dependency recognition, and execution order determination. Synthetic datasets are generated and structural information from control flow and data flow graphs is extracted using Slither. Fine-tuning involves adapter modules based on LoRA, with low-rank normalization fusion enhancing robustness. The combined model achieves 98.2% accuracy, surpassing state-of-the-art, with significant improvements in recall on real contracts.
Key Results
- On synthetic datasets, the LoRA adapters reach 97-99% F1 scores; fused models achieve 94.7% F1 and 98.2% accuracy, outperforming existing methods.
- Evaluation on 31 real-world contracts yields an 87.1% recall, 20% higher than Slither's best performance.
- Structural information and factor fusion contribute to model stability, with AUROC and AUPRC curves nearly overlapping, confirming robustness.
Significance
This research addresses the challenge of limited data and complex structural reasoning in smart contract vulnerability detection. By decomposing tasks into atomic subtasks and leveraging structural cues, the approach significantly improves out-of-distribution generalization and detection accuracy. It bridges the gap between static analysis and deep learning, offering a scalable framework for automated security auditing. The methodology enhances the interpretability and robustness of models, paving the way for safer blockchain ecosystems and broader AI applications in security-critical domains.
Technical Contribution
The paper introduces a novel post-training framework combining atomic task decomposition, structural feature extraction, and factor fusion via low-rank normalization. It guarantees full-rank Jacobian properties and consistency, providing theoretical foundations for robust out-of-distribution generalization. The integration of CFG/DFG cues with adapter-based fine-tuning represents a significant step beyond traditional static analysis and monolithic deep models, enabling scalable, interpretable, and high-performance vulnerability detection.
Novelty
This is the first work applying atomic task decomposition and factor fusion to smart contract reentrancy detection within LLMs. It innovatively combines structural information extraction with multi-task adapter training, achieving superior generalization in data-scarce scenarios. Unlike prior methods relying solely on surface heuristics or rule-based analysis, this approach emphasizes a principled, theoretically grounded decomposition, setting a new benchmark in code security AI.
Limitations
- Dependence on static analysis tools like Slither may limit robustness against obfuscated or highly complex code structures. The synthetic datasets, while extensive, may not fully capture real-world variability, affecting generalization in some edge cases.
- Training and fusion processes are computationally intensive, posing challenges for real-time deployment in resource-constrained environments.
- The approach primarily targets reentrancy vulnerabilities; extending to other vulnerability types requires additional task decomposition and dataset construction.
Future Work
Future directions include integrating multi-modal data such as bytecode and execution traces, expanding to other vulnerability categories, and developing real-time detection systems. Enhancing model interpretability and reducing computational overhead are also key goals. Additionally, exploring federated learning for cross-organizational security models could further improve scalability and privacy-preserving capabilities.
AI Executive Summary
The rapid growth of blockchain technology has made smart contracts a cornerstone of decentralized finance, yet their security remains a critical concern. Reentrancy vulnerabilities, one of the earliest and most notorious flaws, have led to significant financial losses, exemplified by the DAO attack. Traditional static analysis tools like Slither and Mythril have been instrumental in vulnerability detection but struggle with complex, structurally diverse contracts. Meanwhile, large language models (LLMs) have demonstrated impressive natural language understanding but face challenges in specialized domains due to limited training data and structural reasoning requirements.
This paper introduces a novel approach that bridges this gap by decomposing the complex reentrancy detection task into four atomic subtasks: identifying external calls, detecting state updates, recognizing data dependencies, and determining execution order. These subtasks are trained separately on synthetic datasets generated through expert-guided synthesis and structural cues extracted via Slither. Fine-tuning involves adapter modules based on Low-Rank Adaptation (LoRA), with a fusion mechanism that combines subtask outputs into a robust, unified model.
Experimental results show that this method achieves an accuracy of 98.2% in reentrancy detection, outperforming existing state-of-the-art techniques. On real-world contracts, the recall rate exceeds 87%, surpassing traditional static analyzers by over 20%. The structural information from CFG and DFG graphs plays a crucial role in enhancing model interpretability and robustness. The approach not only advances the theoretical understanding of compositional generalization in deep models but also offers practical tools for automated, reliable smart contract auditing.
Looking ahead, the integration of multi-modal data, real-time detection capabilities, and federated learning frameworks will further enhance the scalability and applicability of this technology. The work paves the way for safer blockchain ecosystems, reducing financial risks and fostering broader adoption of decentralized applications.
Deep Dive
Limitations & Outlook
What gaps remain?
Abstract
Large language models (LLMs) demonstrate remarkable capabilities in natural language understanding and generation. Despite being trained on large-scale, high-quality data, LLMs still fail to outperform traditional static analysis tools in specialized domains like smart contract vulnerability detection. To address this issue, this paper proposes a post-training algorithm based on atomic task decomposition and fusion. This algorithm aims to achieve combinatorial generalization under limited data by decomposing complex reasoning tasks. Specifically, we decompose the reentrancy vulnerability detection task into four linearly independent atomic tasks: identifying external calls, identifying state updates, identifying data dependencies between external calls and state updates, and determining their data flow order. These tasks form the core components of our approach. By training on synthetic datasets, we generate three compiler-verified datasets. We then employ the Slither tool to extract structural information from the control flow graph and data flow graph, which is used to fine-tune the LLM's adapter. Experimental results demonstrate that low-rank normalization fusion with the LoRA adapter improves the LLM's reentrancy vulnerability detection accuracy to 98.2%, surpassing state-of-the-art methods. On 31 real-world contracts, the algorithm achieves a 20% higher recall than traditional analysis tools.