StealthGraph: Exposing Domain-Specific Risks in LLMs through Knowledge-Graph-Guided Harmful Prompt Generation
Proposes StealthGraph, using knowledge graphs to generate implicit harmful prompts; employs dual rewriting strategies to enhance stealth, achieving 84.92% attack success.
Key Findings
Methodology
This approach leverages knowledge graphs to extract core domain entities, guiding large language models (LLMs) to generate explicit harmful prompts based on few-shot exemplars. Subsequently, it applies two strategies—direct rewriting and context-enhanced rewriting—to covertly transform explicit prompts into implicit variants, filtering for semantic consistency and fluency. The entire pipeline is automated, producing high-quality, stealthy datasets that cover broad domain risks, supporting safety evaluation and adversarial research.
Key Results
- Across medicine, finance, law, and education, the generated implicit prompts achieved attack success rates of up to 84.92% on models like GPT-4 and Gemini, vastly outperforming public benchmarks (5-23%), demonstrating high effectiveness in real-world scenarios.
- By guiding generation with knowledge graphs, 192 high-risk entities were identified, ensuring comprehensive domain coverage and risk diversity.
- The dual rewriting strategies significantly improved prompt stealthiness while maintaining semantic fidelity, with perplexity around 80, ensuring practical usability.
Significance
This work addresses the critical gap in safety evaluation by systematically generating covert risk prompts that evade surface-level defenses. It enhances understanding of model vulnerabilities in realistic settings, facilitating the development of more robust safety mechanisms. The integration of structured domain knowledge via knowledge graphs provides a scalable, adaptable framework for multi-domain risk assessment, advancing both academic research and industry safety standards.
Technical Contribution
The core innovation lies in combining knowledge graph-guided explicit prompt generation with dual strategy obfuscation to produce highly stealthy harmful prompts. This end-to-end pipeline surpasses traditional keyword-based or single-method approaches, offering a scalable, domain-agnostic solution with high coverage and stealth. It also introduces a reusable dataset for safety testing, enabling standardized evaluation and further research.
Novelty
This is the first systematic use of knowledge graphs to guide the automated synthesis of covert domain-specific harmful prompts, coupled with dual rewriting strategies to maximize stealth. Unlike prior work focused on surface-level jailbreaks or keyword obfuscation, this approach models complex, realistic threat scenarios, representing a significant paradigm shift in safety evaluation methodology.
Limitations
- Dependence on the quality and completeness of knowledge graphs; incomplete or biased data may limit coverage.
- Detection of these covert prompts remains challenging, as models may adapt or develop new strategies to bypass rewriting.
- Computational costs are high, especially for large-scale multi-domain applications, requiring further optimization.
Future Work
Future directions include integrating multi-modal knowledge sources, employing graph neural networks for better entity reasoning, and expanding to more complex, multi-turn scenarios. Enhancing efficiency and robustness against adaptive models will be key, along with developing standardized benchmarks for covert prompt detection.
AI Executive Summary
As large language models (LLMs) become integral to high-stakes fields like healthcare, finance, and law, their safety risks also escalate. Traditional safety measures rely heavily on explicit prompt filtering, which is increasingly insufficient against sophisticated, covert threats. This paper introduces StealthGraph, a novel framework that leverages knowledge graphs to systematically generate domain-specific harmful prompts. By extracting core entities from structured knowledge bases like Wikidata, the system guides LLMs to produce explicit prompts that are then transformed into covert variants through dual rewriting strategies—direct and context-enhanced—making detection significantly more difficult.
The process begins with constructing a domain-specific subgraph, selecting high-frequency, high-risk entities, and using few-shot exemplars to steer prompt generation. The generated explicit prompts are filtered for toxicity and fluency, ensuring quality. Next, the dual rewriting strategies—one directly rephrasing harmful content into more natural, indirect forms, and another enriching prompts with domain-context cards—produce stealthier variants. These are further filtered and validated, resulting in datasets that reflect realistic, covert threats.
Experimental results across four domains show that the implicit prompts achieve attack success rates exceeding 80%, far surpassing existing benchmarks. The approach demonstrates robustness across multiple models, including GPT-4 and proprietary systems, and maintains acceptable fluency levels. This work significantly advances the capability to evaluate and improve LLM safety by exposing hidden vulnerabilities that surface-level defenses cannot detect.
Looking ahead, integrating multi-modal knowledge sources and optimizing computational efficiency will be crucial. The framework offers a scalable, domain-agnostic solution, laying a foundation for future research in covert threat detection and safety mechanism development, ultimately contributing to safer deployment of AI in sensitive areas.
Deep Dive
Abstract
Large language models (LLMs) are increasingly applied in specialized domains such as finance and healthcare, where they introduce unique safety risks. Domain-specific datasets of harmful prompts remain scarce and still largely rely on manual construction; public datasets mainly focus on explicit harmful prompts, which modern LLM defenses can often detect and refuse. In contrast, implicit harmful prompts-expressed through indirect domain knowledge-are harder to detect and better reflect real-world threats. We identify two challenges: transforming domain knowledge into actionable constraints and increasing the implicitness of generated harmful prompts. To address them, we propose an end-to-end framework that first performs knowledge-graph-guided harmful prompt generation to systematically produce domain-relevant prompts, and then applies two-strategy obfuscation rewriting to convert explicit harmful prompts into implicit variants via direct and context-enhanced rewriting. This framework yields high-quality datasets combining strong domain relevance with implicitness, enabling more realistic red-teaming and advancing LLM safety research. We release our code and datasets on GitHub.