The Effect of Multi-Lingual and Keyword Adversarial Injection on LLM Relevance Judgment
This study evaluates multilingual keyword and content variant injection attacks on LLM relevance judgments, revealing persistent inflation and evasion of defenses.
Key Findings
Methodology
Using TREC Deep Learning 2022 data, two open-source models (GPT-OSS-20B and Qwen-32B) were tested under instruction and content injection strategies across 8 languages with diverse resource levels. Adversarial content included keywords, variants, and distracting passages. Defense mechanisms like PromptArmor were employed to evaluate mitigation effectiveness. Metrics such as false positive (FP) and false negative (FN) rates quantified bias. Statistical analysis via two-way ANOVA examined the interaction between model type and language, confirming the transferability and robustness of multilingual content-based attacks.
Key Results
- Multilingual keyword and content injections consistently increased FP rates across all models and languages, with some reaching 35%. Content variants and distractors effectively bypassed defenses, maintaining high attack efficacy. Even with PromptArmor, many injections persisted, indicating limitations in current filtering approaches. Qwen-32B showed lower FP but higher FN, reflecting a conservative judgment style. The statistical significance of model-language interaction (p<0.001) underscores the vulnerability of multilingual models to content-based adversarial attacks.
- Simple transformations like query variants and distractor passages reliably evade detection, demonstrating high transferability. The experiments reveal that existing defenses are insufficient against complex, multilingual content manipulations. These findings highlight the critical need for proactive, multi-layered defense strategies to safeguard LLM-based relevance judgments in real-world, multilingual settings.
- Overall, the results emphasize that current models are susceptible to subtle, cross-lingual content manipulations, which can distort evaluation outcomes and undermine trust in automated relevance judgments. The robustness gap calls for innovative defense mechanisms tailored to multilingual environments, including semantic-aware filtering and adversarial training.
Significance
This research exposes a significant vulnerability in LLM-based relevance judgment systems, especially in multilingual contexts. As these models increasingly serve as automated evaluators in information retrieval, their susceptibility to content-based adversarial attacks poses risks to evaluation integrity and system reliability. The findings demonstrate that existing defenses, such as surface-level keyword filtering, are inadequate against sophisticated, multilingual content manipulations. Addressing this gap is crucial for deploying trustworthy, scalable, and fair AI systems in global information environments. The study advocates for developing more proactive, semantic-aware, and multilingual-aware defense strategies, which are vital for future research and practical deployment of robust LLM evaluators.
Technical Contribution
This work pioneers a comprehensive evaluation of multilingual content-based adversarial injections targeting LLM relevance judgments. It combines diverse attack strategies—keywords, variants, distractors—with cross-lingual testing across multiple models and resource levels. The integration of statistical analysis confirms the significance of language-model interactions, revealing transferability and robustness issues. The study proposes enhancements to existing filtering mechanisms like PromptArmor, incorporating semantic understanding and adaptive detection. These contributions advance the understanding of multilingual adversarial robustness, providing a foundation for designing resilient evaluation frameworks and defense mechanisms in multi-language NLP systems.
Novelty
This is the first systematic investigation into how multilingual content variants and distractors can be used as adversarial attacks against LLM relevance judgments. Unlike prior work focusing on single-language or instruction-based attacks, this study emphasizes content-based manipulations across diverse languages and resource levels. It demonstrates the transferability of such attacks and evaluates the limitations of current defenses, highlighting a critical gap in multilingual robustness. The innovative combination of content variants, distractors, and statistical analysis offers new insights into the vulnerabilities of multilingual LLMs, setting a foundation for future security-focused research.
Limitations
- The experiments are limited to the TREC-DL dataset, which may not fully represent real-world, noisy, or domain-specific scenarios. The defense mechanisms tested are primarily rule-based or model-centric, which might not generalize well to unseen attack strategies.
- The transferability of content-based injections in extremely low-resource languages or very long documents remains to be validated. FN rates in some languages suggest that model calibration and fine-tuning are necessary for broader applicability.
- Future work should explore multi-modal defenses, adversarial training, and real-time detection to improve robustness. Additionally, expanding to more diverse datasets and real-world applications will be essential for practical deployment.
Future Work
Future research should focus on developing semantic-aware, proactive defense frameworks that can detect complex, multilingual content manipulations. Investigating adversarial training, multi-modal approaches, and continuous evaluation platforms will be critical. Extending analysis to downstream tasks like retrieval-augmented generation (RAG) and human evaluation under adversarial conditions will deepen understanding. Building comprehensive, multilingual robustness benchmarks and deploying adaptive defenses in real-world systems are vital steps toward trustworthy AI in global information environments.
AI Executive Summary
The rapid adoption of large language models (LLMs) as automated relevance evaluators in information retrieval has raised concerns about their robustness against adversarial manipulations. While prior work primarily focused on instruction-based prompt injections, recent findings suggest content-based attacks, especially in multilingual settings, pose a significant threat. This study systematically investigates how multilingual keyword and content variants can be used to inflate relevance scores and evade existing defenses.
Using the TREC Deep Learning 2022 dataset, two open-source models—GPT-OSS-20B and Qwen-32B—were evaluated across 8 languages with diverse resource levels. Attack strategies included keyword insertion, query variants, and distractor passages, all designed to mimic natural language while misleading the models. Results showed persistent false-positive inflation, with some attack variants achieving FP rates up to 35%, even when applying filtering mechanisms like PromptArmor. The statistical analysis confirmed that both model type and language significantly influence the effectiveness of these attacks, with notable interaction effects.
These findings reveal a critical vulnerability: content-based, multilingual adversarial injections can reliably manipulate relevance judgments across different models and languages. Such manipulations threaten the integrity of automated evaluation pipelines, potentially leading to biased system development and evaluation distortion. The research underscores the urgent need for more sophisticated, proactive defense strategies that go beyond surface-level filtering, incorporating semantic understanding and multi-layered detection.
In conclusion, as LLMs become central to multilingual information retrieval, ensuring their robustness against complex, cross-lingual adversarial content is paramount. This work provides a foundational step toward developing resilient evaluation frameworks, emphasizing the importance of continuous, adversarial-aware testing. Future efforts should focus on integrating semantic-aware defenses, adversarial training, and real-world deployment scenarios to safeguard the reliability and fairness of AI-driven relevance judgments in a globalized digital landscape.
Deep Analysis
Background
Recent advances in large language models (LLMs) such as BERT, GPT-3, and their open-source counterparts have transformed information retrieval and relevance assessment. These models leverage deep neural architectures and pretraining on massive multilingual corpora to achieve cross-lingual transferability. Prior studies, including Dietz et al. (2025) and Thomas et al. (2025), demonstrated that LLMs can approximate human relevance judgments with high accuracy, reducing reliance on manual annotation. However, as these models are increasingly deployed in automated evaluation pipelines, their security vulnerabilities have come under scrutiny. Early research on adversarial prompting (Li et al., 2025) focused on instruction-based prompt injections, which can mislead models by overriding their initial instructions. Nonetheless, content-based manipulations—such as inserting keywords, variants, or distractor passages—remain underexplored, especially in multilingual contexts. The challenge is compounded by models’ strong cross-lingual capabilities, which enable attackers to craft multilingual content that bypasses language-specific filters, raising concerns about robustness and fairness in global applications.
Core Problem
The core issue addressed in this work is the vulnerability of LLM-based relevance judgments to multilingual, content-based adversarial injections. Existing defenses like keyword filtering or prompt-based detection are insufficient against sophisticated content manipulations that adapt across languages. These attacks inflate relevance scores or cause misclassification, undermining evaluation integrity. The problem is particularly pressing in multilingual settings, where models’ cross-lingual transferability facilitates attack transferability. This poses a significant challenge for deploying reliable, fair, and secure AI systems in global information retrieval, requiring new strategies to detect and mitigate such complex, cross-lingual adversarial content.
Innovation
This study introduces several innovations: 1) First, it systematically evaluates the impact of multilingual keyword and content variants on LLM relevance judgments, revealing their transferability and effectiveness across diverse languages. 2) It combines multiple attack strategies—keywords, query variants, distractor passages—and assesses their evasion of existing defenses like PromptArmor, highlighting the limitations of surface-level filtering. 3) It employs statistical analysis (two-way ANOVA) to quantify the interaction effects between model type and language, providing a theoretical understanding of vulnerabilities. 4) It proposes enhancements to filtering mechanisms, integrating semantic understanding and adaptive detection, to improve robustness against multi-language content manipulations. These innovations collectively advance the understanding of multilingual adversarial robustness in LLM evaluation systems.
Methodology
- �� Data collection: Utilized TREC Deep Learning 2022 dataset, covering multiple languages with relevance judgments.
- �� Models: Employed GPT-OSS-20B and Qwen-32B, both open-source, evaluated under instruction and content injection scenarios.
- �� Attack design: Created keyword, variant, and distractor content injections, covering 8 languages with diverse scripts and resource levels.
- �� Defense mechanisms: Applied rule-based keyword filters and PromptArmor, analyzing their effectiveness.
- �� Evaluation metrics: Measured false positive (FP) and false negative (FN) rates, comparing injected versus baseline judgments.
- �� Statistical analysis: Conducted two-way ANOVA to assess the significance of model and language effects, validating transferability and interaction effects.
Experiments
Experiments involved injecting content into queries and passages across multiple languages, then measuring the impact on relevance judgments. The models' outputs were compared with human judgments to calculate FP and FN rates. Different attack variants, including keyword insertion, query paraphrasing, and distractor passages, were tested with and without filtering. The robustness of defenses like PromptArmor was evaluated by their ability to detect and remove injected content. Cross-lingual transferability was assessed by applying attacks in resource-rich and low-resource languages. The experimental setup included multiple runs to ensure statistical significance, with hyperparameters tuned for optimal attack strength and detection sensitivity. Results were analyzed using ANOVA and other statistical tests to confirm the significance of observed effects.
Results
The experiments demonstrated that all attack strategies significantly increased FP rates, with some reaching 35%, across models and languages. Content variants and distractors effectively bypassed defenses, maintaining high attack success rates. PromptArmor reduced the impact of instruction-based injections but was less effective against content-based variants, especially in low-resource languages. Statistical analysis confirmed that both model type and language significantly influence attack efficacy (p<0.001), with notable interaction effects. These findings reveal that current defenses are inadequate against sophisticated, multilingual content manipulations, emphasizing the need for semantic-aware, proactive defense strategies.
Applications
The findings are directly applicable to multilingual search engines, automated relevance evaluators, and content filtering systems, where robustness against adversarial content is critical. Enhancing models with semantic understanding and adaptive detection can improve evaluation fairness and reliability. These insights also inform the development of security protocols for AI systems deployed in diverse linguistic environments, ensuring trustworthy and unbiased information retrieval. Long-term, integrating adversarial training and multi-modal defenses could lead to resilient, scalable evaluation frameworks suitable for global deployment.
Limitations & Outlook
The study is limited to the TREC-DL dataset, which may not encompass all real-world complexities. The defense mechanisms tested are primarily rule-based or model-centric, potentially less effective against unseen or more sophisticated attacks. Transferability of content injections in extremely low-resource languages and very long documents remains to be validated. FN rates in some languages suggest the need for more fine-grained calibration. Future work should explore multi-modal defenses, adversarial training, and broader datasets to improve robustness and generalizability.
Plain Language Accessible to non-experts
Imagine a big factory where workers from many different countries assemble toys. Each worker speaks a different language and has their own way of doing things. Sometimes, some workers try to sneak in fake parts or hidden messages into the toys to trick the inspectors (the models). The inspectors are supposed to check if the toys are good, but if they only look for obvious signs, sneaky workers can hide their tricks in ways that are hard to detect, especially when they speak different languages. This makes it difficult for the inspectors to tell real toys from fake ones. The factory needs smarter inspectors who understand all languages and can spot hidden tricks, no matter how sneaky the workers are. This story shows how tricky it is to keep things honest when people try to cheat in many different languages, and why we need better ways to catch them.
ELI14 Explained like you're 14
Imagine you're playing a game with friends from all over the world. Some friends want to cheat by hiding secret messages in their words so you think they're telling the truth, but actually, they're sneaking in tricks. If you only look for obvious signs, they might still fool you, especially if they speak different languages or change their words a little. To catch these sneaky tricks, you need to be really clever—like knowing all the languages and understanding how words can be changed without losing their meaning. That way, you can spot the tricks no matter how tricky they are. This is similar to how computers try to check if information is real or fake when there are many languages and sneaky tricks involved. We need smarter computers to keep things honest in a world full of different languages and tricky words!
Abstract
Large language models (LLMs) are increasingly being used as automated judges for relevance evaluation in information retrieval, yet their robustness to adversarial manipulation remains insufficiently understood, particularly in multilingual settings. In this work, we investigate the impact of cross-lingual prompt injection attacks on LLM-based relevance judgments using TREC Deep Learning collections and two open-weight models under established prompting frameworks. We examine both instruction-based and content-based injection strategies in 8 languages spanning different resource levels. Our results demonstrate that multilingual query-based injections are highly effective in inflating relevance scores while simultaneously evading existing prompt-injection defenses. We further found that, although existing defense mechanisms can be modified to mitigate such attacks, these injections can be easily adapted to bypass them. These findings highlight a critical gap in current defense approaches and demonstrate that language generalization can act as an attack vector, underscoring the need for more robust and proactive evaluation frameworks for LLM-as-a-judge systems.