The Language Blind Spot: How Query Language and Brand Recognition Tier Shape AI-Constructed Brand Reputation Across Twelve European Languages
Using BGE-M3 embeddings, analysis of 66 brands across 12 European languages reveals significant language bias in AI-generated brand reputation, with model choice affecting stability more than language.
Key Findings
Methodology
This study employed three grounded large language models (GPT-5.4, Gemini 3.1 Pro, Perplexity Sonar Pro) to query 66 brands across eleven European markets in twelve languages spanning four language families. A total of 35,640 responses were generated. Multilingual embeddings (BGE-M3) enabled direct semantic comparisons across languages without translation. The experimental design was a fully crossed factorial setup, varying language, model, brand, and prompt templates. Analyses included cosine similarity for semantic divergence, sentiment scoring with XLM-RoBERTa, recommendation share metrics, source citation classification, hierarchical clustering for language family structure, and principal component analysis for brand visibility dimensions. Statistical tests such as ANOVA, t-tests, and chi-square were used to evaluate differences and correlations.
Key Results
- AI-constructed brand reputation exhibits strong language dependence: the average cross-language cosine similarity is 0.825. Responses within the same language family are more similar (0.844) than across families (0.820), with a significant t-value of 57.98 (p<0.001, d=0.31). Sentiment varies systematically by language, with Uralic and Baltic languages showing more positive sentiment (+0.094 and +0.055 respectively), whereas Germanic languages, including English, are more critical (−0.060 and −0.147). Hierarchical clustering accurately recovers language family groupings (cophenetic=0.915).
- Query language impacts brand recommendation rates more than descriptive sentiment: shifting from English to a brand’s home language increases recommendation share by 0.80 for local champions, but only by 0.15 for multinationals. No comparable reversal occurs in sentiment. This indicates that English-only monitoring underestimates local brand visibility, creating a language blind spot.
- Response stability is more influenced by model choice than language: across five repetitions on a subset of 20 brands, model differences explained η²=0.32 of variance, versus η²=0.01 for language. Principal component analysis suggests that AI brand visibility primarily resides along two axes—source breadth and prominence—with a weaker third dimension. These findings underscore the importance of multi-model, multi-language approaches for reliable AI reputation measurement.
Significance
This research highlights a critical blind spot in current AI-driven brand reputation monitoring: reliance on English-only data overlooks significant variations in local language contexts. The findings demonstrate that local brands are substantially more visible in their native languages, which has implications for global marketing strategies and AI fairness. By quantitatively establishing the extent of language bias, the study informs the development of more equitable and comprehensive AI monitoring systems. It addresses a long-standing challenge in multilingual NLP—how to accurately capture cross-cultural differences in AI outputs—by providing empirical evidence and methodological frameworks that can be adopted in industry and academia.
Technical Contribution
The paper introduces a novel framework combining multilingual embeddings (BGE-M3) with a fully crossed experimental design involving three grounded LLMs, enabling precise cross-language semantic and sentiment comparisons. It innovates by applying hierarchical clustering to language responses, revealing structural relationships aligned with language families, and employing principal component analysis to decompose AI brand visibility into interpretable dimensions. The integration of recommendation share, source citation classification, and stability metrics offers a comprehensive toolkit for evaluating AI reputation bias and robustness across languages and models, advancing the state-of-the-art in multilingual AI evaluation.
Novelty
This is the first large-scale, systematic investigation into how query language influences AI-generated brand reputation across multiple languages, models, and brands. Unlike prior work focusing on model biases or performance metrics, this study emphasizes the semantic and perceptual dimensions of AI outputs in a multilingual context. Its core innovation lies in quantifying the language-dependent divergence of AI narratives and recommendation patterns, providing a new perspective on fairness and bias in AI-mediated reputation management.
Limitations
- The study measures AI outputs without direct validation against human perceptions or market data, so it remains uncertain whether observed divergences reflect real-world reputation differences. Incorporating consumer surveys would strengthen the findings.
- Sentiment analysis based on XLM-RoBERTa may be biased by the model’s tendency to interpret long, structured responses negatively, which could distort absolute sentiment scores. Relative comparisons are more reliable.
- Stability analysis was limited to a subset of 20 brands; the main dataset’s response consistency over multiple queries remains unverified. Future work should extend stability assessments to larger samples.
Future Work
Future research should integrate consumer perception data to validate whether AI output biases correspond to actual brand reputations in different markets. Developing multilingual, fairer language models with reduced bias is essential. Additionally, exploring multimodal approaches—combining textual, visual, and contextual data—could enhance the robustness of AI reputation assessments. Long-term, establishing standardized benchmarks for cross-lingual AI fairness and stability will be crucial for deploying trustworthy reputation monitoring systems globally.
AI Executive Summary
In today’s interconnected digital economy, brand reputation management increasingly relies on AI technologies, especially large language models (LLMs), which serve as mediators between organizations and their stakeholders. These models generate narratives, recommendations, and sentiment signals that influence consumer perceptions and strategic decisions. However, most existing monitoring efforts are predominantly conducted in English, under the assumption that an English query yields a representative picture of brand perception across diverse markets. This assumption, if invalid, can lead to significant blind spots, particularly in multilingual regions such as Europe.
This study by Dmitrij Żatuchin critically examines the extent to which query language influences AI-constructed brand reputation across twelve European languages, spanning four language families. By deploying three grounded LLMs—GPT-5.4, Gemini 3.1 Pro, and Perplexity Sonar Pro—across 66 brands in eleven countries, the research generates a comprehensive dataset of 35,640 responses. The core innovation lies in leveraging BGE-M3 multilingual embeddings, which allow direct semantic comparisons across languages without translation, thus avoiding translation-induced biases.
The findings reveal that AI-generated brand narratives are significantly language-dependent. The average cross-language semantic similarity is 0.825, with responses within the same language family being more similar than those across families. Sentiment analysis shows a systematic variation: Uralic and Baltic languages tend to produce more positive sentiments, while Germanic languages, including English, are more critical. Hierarchical clustering of responses accurately recovers language family structures, confirming the linguistic basis of narrative divergence.
More critically, the study uncovers that query language dramatically affects which brands are recommended. Shifting from English to a brand’s native language increases the recommendation share for local champions by 0.80, whereas global multinationals see only a 0.15 increase. This suggests that English-only monitoring underrepresents local brands’ visibility, creating a ‘language blind spot’ that disproportionately affects smaller, non-English-speaking markets.
Response stability analysis indicates that the choice of model influences output consistency more than language. Across multiple repetitions, differences between models explained a much larger portion of variance than language differences. Principal component analysis further decomposes AI brand visibility into two primary axes—source breadth and prominence—with a weaker third dimension, providing a nuanced understanding of how AI constructs brand reputation.
These insights have profound implications for both academia and industry. They highlight the necessity of multi-language, multi-model approaches to accurately monitor and manage brand reputation in multilingual environments. The research advocates for more inclusive AI systems that recognize linguistic and cultural diversity, thereby reducing bias and improving fairness.
Despite its strengths, the study acknowledges limitations, including the lack of direct validation against human perception and the potential biases of sentiment analysis models. Future work should incorporate consumer surveys, develop fairer multilingual models, and explore multimodal data integration. Overall, this research advances our understanding of AI-mediated reputation, emphasizing the importance of linguistic inclusivity in AI systems to ensure fair and comprehensive brand monitoring worldwide.
Deep Dive
Abstract
Large language models (LLMs) increasingly mediate how people form impressions of organisations, yet most monitoring is done in English, assuming an English query returns a representative picture. We measure how far that holds. We queried three grounded LLMs (GPT-5.4, Gemini 3.1 Pro, Perplexity Sonar Pro) about 66 brands from eleven Northern, Baltic, and Central European markets, in twelve languages across four families (Germanic, Uralic, Baltic, Slavic), generating 35,640 responses. Multilingual embeddings (BGE-M3) allow cross-language comparison without translation. Three results emerge. First, AI-constructed reputation is language-bound: mean cross-language cosine similarity is 0.825, same-family responses are more similar than cross-family (0.844 vs 0.820; d = 0.31), and sentiment varies by language (F = 268.5, eta^2 = 0.077), with Uralic and Baltic languages most positive and Germanic, including English, most critical; clustering recovers the Slavic and Baltic families (cophenetic 0.915). Second, query language shifts which brands are recommended far more than how they are described: moving from an English query to a brand's home language raises recommendation share by 0.80 for local champions but only 0.15 for global multinationals (t = -8.84, p < 0.001), with no comparable reversal in sentiment. An English-only audit therefore understates a local champion's AI visibility. Third, response stability varies more with model choice than with language (eta^2_model = 0.32 vs eta^2_language = 0.01, on a five-iteration replication over a 20-brand subset). These results indicate that English-only AI reputation monitoring leaves a measurable language blind spot, concentrated in the visibility of locally headquartered brands.
References (20)
Who Owns the AI Recommendation? A Multi-Industry Empirical Map of Brand Category Ownership Across Large Language Models
D. Żatuchin
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Emily M. Bender, Timnit Gebru, Angelina McMillan-Major et al.
Essentials of Corporate Communication
C. V. Riel, C. Fombrun
Agenda setting function of mass media
M. McCombs
What's in a Name? Reputation Building and Corporate Strategy
C. Fombrun, Mark T. Shanley
Corporate Citizenship: Creating Intangible Assets Across Institutional Environments
Naomi A. Gardberg, C. Fombrun
Stakeholder Tracking and Analysis: The RepTrak® System for Measuring Corporate Reputation
C. Fombrun, Leonard J. Ponzi, William Newburry
The effects of institutional development and national culture on cross-national differences in corporate reputation
David L. Deephouse, William Newburry, Abrahim Soleimani
Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
Tolga Bolukbasi, Kai-Wei Chang, James Y. Zou et al.
How Multilingual is Multilingual BERT?
Telmo Pires, Eva Schlinger, Dan Garrette
Language Models as Knowledge Bases?
F. Petroni, Tim Rocktäschel, Patrick Lewis et al.
Unsupervised Cross-lingual Representation Learning at Scale
Alexis Conneau, Kartikay Khandelwal, Naman Goyal et al.
Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus
Jesse Dodge, Ana Marasovic, Gabriel Ilharco et al.
No Language Left Behind: Scaling Human-Centered Machine Translation
Nllb team, M. Costa-jussà, James Cross et al.
Large Language Models Struggle to Learn Long-Tail Knowledge
Nikhil Kandpal, H. Deng, Adam Roberts et al.
When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories
Alex Troy Mallen, Akari Asai, Victor Zhong et al.
MEGA: Multilingual Evaluation of Generative AI
Kabir Ahuja, Rishav Hada, Millicent Ochieng et al.
MMLU-ProX: A Multilingual Benchmark for Advanced Large Language Model Evaluation
Weihao Xuan, Rui Yang, Heli Qi et al.
Lost in Multilinguality: Dissecting Cross-lingual Factual Inconsistency in Transformer Language Models
Mingyang Wang, Heike Adel, Lukas Lange et al.
Tracing Multilingual Factual Knowledge Acquisition in Pretraining
Yihong Liu, Mingyang Wang, Amir Hossein Kargaran et al.
Cited By (1)
How Large Language Models Source Brand Reputation Across Languages and Markets