STEREODISCO: Discovering Stereotypicality in LLMs
Proposed STEREODISCO framework applies semantic differential method to discover and quantify stereotypical axes in LLM internal representations, revealing new bias dimensions beyond prior social psychology research.
Key Findings
Methodology
The approach adapts the semantic differential method (Osgood et al., 1957) to internal LLM representations by constructing approximately 2000 candidate semantic axes from WordNet antonym synsets. It involves generating labeled sentence prompts for each antonym pair, probing model activations across multiple layers and attention heads, and mapping these activations linearly into geometric axes. Concept projections are obtained by embedding concept prompts and projecting onto these axes. A statistical Kolmogorov–Smirnov test then determines whether concept projections differ significantly from a reference set, indicating stereotypical axes. The framework's components include axis sourcing, geometric mapping, concept projection, and significance testing.
Key Results
- Experiments with LLAMA-3-8B-INSTRUCT and MISTRAL-7B-INSTRUCT revealed high inter-model agreement (correlation 0.75) on social group stereotypes, surpassing their alignment with human ratings (correlations 0.55–0.63). This suggests internal biases are consistent across models but diverge from human social perceptions.
- Novel stereotypical axes such as humble vs. proud, narrow-minded vs. broad-minded, and cowardly vs. brave were identified, with independent human annotations confirming their stereotype status. These axes extend beyond traditional social psychology axes, highlighting previously unexamined biases.
- The statistical testing identified 50 axes with significant stereotypicality (p<0.05). The findings demonstrate the presence of rich, quantifiable biases encoded in model activation spaces, with implications for bias mitigation and interpretability.
Significance
This work advances understanding of bias in LLMs by moving beyond surface-level word embedding analysis to internal activation geometry. It provides a scalable, systematic method for discovering hidden stereotypical axes, offering insights into model fairness and transparency. The identification of new bias dimensions informs future mitigation strategies, contributing to ethical AI deployment. The approach bridges social psychology and deep learning, enabling more nuanced bias detection and control in large models.
Technical Contribution
The paper introduces a novel geometric probing framework that maps model activations onto interpretable semantic axes, combining large-scale candidate generation with statistical validation. It leverages attention head activations for precise internal localization of semantic axes, surpassing prior methods limited to word embeddings or layer-averaged representations. The framework's modular design allows broad applicability across concept families and bias types, setting a new standard for internal bias detection in LLMs.
Novelty
This is the first comprehensive system that systematically discovers and quantifies semantic axes within LLM activation spaces using a large-scale, theory-grounded approach. Unlike previous work relying on predefined dictionaries or associative tests, it employs a geometric probing method rooted in psychometric principles, enabling discovery of previously unrecognized bias axes. Its integration of statistical testing and internal localization represents a significant methodological leap.
Limitations
- The reliance on WordNet antonym pairs may limit coverage of non-bipolar biases, potentially missing subtle or complex stereotypes. The linear geometric assumption may oversimplify non-linear bias structures. Statistical tests depend on sample size, which could affect detection sensitivity in small concept sets. Future work should incorporate non-linear mappings and broader lexical sources.
- Probing at attention head level, while precise, may be affected by model architecture variations and layer-specific effects, introducing potential biases in axis localization. Additionally, the method's computational cost scales with candidate axes and model size, posing challenges for very large models.
- The approach primarily detects stereotypical associations but does not directly address causality or the dynamics of bias formation. Further research is needed to link internal biases with downstream behaviors and societal impacts.
Future Work
Future directions include extending the framework to multi-modal models integrating visual and textual data, exploring non-linear geometric mappings for richer bias detection, and developing bias mitigation techniques based on identified axes. Additionally, integrating human-in-the-loop validation and causal analysis could deepen understanding of bias origins. Scaling the approach to larger models and diverse concept categories will also be pursued to enhance robustness and coverage.
AI Executive Summary
Large language models (LLMs) have revolutionized natural language processing, yet their internal biases and stereotypes pose significant ethical challenges. Traditional methods for bias detection, such as word embedding association tests, offer limited insights into the internal structure of these models. Recognizing this gap, the authors introduce STEREODISCO, a novel framework that adapts the semantic differential method—originally from social psychology—to probe the internal activation space of LLMs. This approach constructs thousands of candidate semantic axes from WordNet antonym pairs, maps model activations linearly into geometric axes, and employs statistical tests to identify axes that encode stereotypical associations.
The core innovation lies in leveraging attention head activations across multiple layers, enabling precise internal localization of semantic axes. By projecting concepts onto these axes, the framework quantifies their stereotypicality, revealing biases that extend beyond traditional social psychology axes. Experiments on instruction-tuned models LLAMA-3-8B-INSTRUCT and MISTRAL-7B-INSTRUCT show high consistency between models and significant divergence from human stereotypes, indicating that models encode internal biases that are not always aligned with societal perceptions.
Remarkably, the study uncovers new bias dimensions, such as humble vs. proud and cowardly vs. brave, validated through independent human annotations. These findings demonstrate the richness of internal biases and the importance of systematic detection tools. The approach offers a scalable, theory-grounded method for bias analysis, with broad implications for model interpretability, fairness, and ethical AI development. Despite limitations related to lexical coverage and linear assumptions, the framework sets a foundation for future multi-modal, non-linear bias detection and mitigation strategies, advancing the goal of fair and transparent AI systems.
Deep Dive
Abstract
LLMs encode, convey, and perpetuate stereotypes. Prior computational research focuses on a small set of semantic axes investigated in social psychology, and operates on word embeddings produced by language models, leaving open which other semantic axes carry stereotypical associations in LLMs and how LLMs internally represent such axes. We introduce STEREODISCO, a framework that adapts the semantic differential method (Osgood et al., 1957) to the systematic study of stereotypes in LLM internal representations. STEREODISCO constructs approx. 2,000 candidate semantic axes from WordNet antonym synsets, recovers each as a geometric axis in the LLM's activation space via probing, and identifies stereotypical axes via a statistical test over concept projections. As a case study, we apply STEREODISCO to social group stereotypes with LLAMA-3-8B-INSTRUCT and MISTRAL-7B-INSTRUCT. We find that the two LLMs agree with each other on social group ratings more than with humans, suggesting that LLM-encoded stereotype content diverges from that documented in social psychology. We also discover stereotypical axes not investigated in prior work -- including humble vs. proud, narrow-minded vs. broad-minded, and cowardly vs. brave, which human annotators independently confirm.