vEcho: A Paradigm Shift from Vulnerability Verification to Proactive Discovery with Large Language Models
vEcho leverages large language models with memory and reasoning to shift from passive vulnerability verification to proactive discovery, achieving 65% detection rate and reducing false positives to 59.78%.
Key Findings
Methodology
vEcho employs deep context-aware verification, a cognitive memory module, and an echoic vulnerability propagation (EVP) mechanism to transform LLMs from passive filters into active security agents. It integrates multi-source scanning, tool invocation (code navigation, project understanding, web search), and pattern learning from verified vulnerabilities. The EVP mechanism leverages similarity inference to proactively generate new audit hypotheses, enabling cross-project knowledge transfer. Experiments on CWE-Bench-Java show detection rate of 65%, a 41.8% improvement over IRIS, with false positive rate reduced to 59.78%.
Key Results
- Achieves a detection rate of 65%, significantly surpassing IRIS’s 45.83%, demonstrating enhanced capability in identifying vulnerabilities.
- Reduces false positive rate to 59.78%, a 28.3% relative decrease from IRIS, effectively alleviating alert fatigue.
- Proactively discovered 51 novel 0-day vulnerabilities in open-source projects, including critical RCE and ACE exploits, validating practical effectiveness.
Significance
This work addresses fundamental limitations of traditional SAST tools, which suffer from high false positives and limited context understanding. By enabling LLMs to actively learn, reason, and propagate vulnerabilities, vEcho significantly advances automated security analysis. It offers a scalable, intelligent approach to reduce manual effort, improve detection accuracy, and proactively uncover unknown threats, thus pushing the frontier of automated software security. Its paradigm shift from passive filtering to active discovery opens new avenues for research and industrial application.
Technical Contribution
The main technical innovation lies in integrating a cognitive memory module with an EVP mechanism into the LLM-based analysis pipeline. This allows knowledge accumulation across projects, pattern generalization, and proactive vulnerability inference. Unlike IRIS, which is confined to rule-based verification, vEcho’s architecture supports continuous learning and hypothesis generation, providing theoretical guarantees for cross-project knowledge transfer and active discovery. Its modular design enables flexible tool invocation and iterative learning, opening new possibilities for scalable, intelligent static analysis.
Novelty
This is the first work to embed a memory-driven, active inference mechanism into LLM-assisted static analysis, transforming the model from a passive filter into an autonomous security expert. The EVP mechanism, combined with project-level context integration, enables proactive and cross-project vulnerability discovery, representing a fundamental innovation over prior rule-based or passive LLM approaches like IRIS. This paradigm shift has significant implications for both research and practical security auditing.
Limitations
- The reliance on large models’ reasoning capabilities may lead to inaccuracies in highly complex or ambiguous code scenarios, posing false positives or negatives.
- Understanding extensive project context remains challenging, especially for very large codebases, potentially affecting verification precision.
- Computational costs for iterative learning and secondary scans are high, requiring further optimization for industrial-scale deployment.
Future Work
Future research will focus on integrating dynamic analysis to complement static reasoning, reducing computational overhead, and expanding multi-language support. Exploring automated vulnerability repair and real-time continuous auditing are promising directions. Additionally, enhancing the interpretability of the model’s reasoning process will be crucial for industrial adoption.
AI Executive Summary
In modern software development, static application security testing (SAST) tools are essential but face significant challenges. Traditional tools like CodeQL and Coverity rely on predefined rules and data flow analysis, which often lead to high false positive rates and missed vulnerabilities. This results in alert fatigue, consuming valuable security resources and risking overlooked threats. Recent advances introduced large language models (LLMs) to assist in vulnerability detection, but these approaches remain largely passive, limited to rule generation or alert filtering, lacking the ability to learn from past experiences or reason across projects.
This paper introduces vEcho, a novel framework that fundamentally redefines the role of LLMs in security analysis. By integrating deep context-aware verification, a cognitive memory module, and an echoic vulnerability propagation (EVP) mechanism, vEcho transforms LLMs into active security agents capable of learning, reasoning, and proactively discovering vulnerabilities. Its architecture includes stages for candidate generation, deep contextual verification, knowledge feedback, and proactive vulnerability propagation. This design allows vEcho to not only verify potential issues with high precision but also to learn from verified vulnerabilities—both true positives and false positives—and infer new, analogous flaws across codebases.
Experimental results on the CWE-Bench-Java dataset demonstrate vEcho’s superiority, with a detection rate of 65%, compared to 45.83% for IRIS, and a false positive rate reduced to 59.78%. More impressively, in real-world open-source projects, vEcho proactively identified 51 new 0-day vulnerabilities, including critical remote code execution (RCE) and arbitrary code execution (ACE) exploits, confirmed by industry partners. These findings underscore the framework’s potential to revolutionize automated security auditing, making it more intelligent, proactive, and scalable.
The significance of this work lies in its paradigm shift: moving from passive, rule-based verification to active, knowledge-driven discovery. This approach not only enhances detection accuracy but also reduces manual effort, enabling scalable, continuous security assessment. While promising, challenges remain, such as computational costs and the need for broader language support. Future directions include integrating dynamic analysis, improving interpretability, and expanding deployment in industrial environments. Overall, vEcho marks a pivotal step toward autonomous, intelligent cybersecurity systems capable of staying ahead of emerging threats.
Deep Dive
Abstract
Static Application Security Testing (SAST) tools often suffer from high false positive rates, leading to alert fatigue that consumes valuable auditing resources. Recent efforts leveraging Large Language Models (LLMs) as filters offer limited improvements; however, these methods treat LLMs as passive, stateless classifiers, which lack project-wide context and the ability to learn from analyses to discover unknown, similar vulnerabilities. In this paper, we propose vEcho, a novel framework that transforms the LLM from a passive filter into a virtual security expert capable of learning, memory, and reasoning. vEcho equips its core reasoning engine with a robust developer tool suite for deep, context-aware verification. More importantly, we introduce a novel Echoic Vulnerability Propagation (EVP) mechanism. Driven by a Cognitive Memory Module that simulates human learning, EVP enables vEcho to learn from verified vulnerabilities and proactively infer unknown, analogous flaws, achieving a paradigm shift from passive verification to active discovery. Extensive experiments on the CWE-Bench-Java dataset demonstrate vEcho's dual advantages over the state-of-the-art baseline, IRIS. Specifically, vEcho achieves a 65% detection rate, marking a 41.8% relative improvement over IRIS's 45.83%. Crucially, it simultaneously addresses alert fatigue by reducing the false positive rate to 59.78%, a 28.3% relative reduction from IRIS's 84.82%. Furthermore, vEcho proactively identified 37 additional known vulnerabilities beyond the 120 documented in the dataset, and has discovered 51 novel 0-day vulnerabilities in open-source projects.