Representational Harms in LLM-Generated Narratives Against Global Majority Nationalities

Large language models (LLMs) are increasingly used for text generation tasks from everyday use to high-stakes enterprise and government applications, including simulated interviews with asylum seekers. However, these technologies are not value-neutral; they may encode and perpetuate harmful biases against non-dominant communities worldwide. To better evaluate and mitigate such harms, more research examining how LLMs portray diverse individuals is needed.

In this study, we analyze how national origin identities are portrayed by widely-adopted LLMs in response to open-ended narrative generation prompts. Our findings demonstrate the presence of persistent representational harms by national origin, including harmful stereotypes, erasure, and one-dimensional portrayals of Global Majority identities. Minoritized national identities are simultaneously underrepresented in power-neutral stories and overrepresented in subordinated character portrayals, which are over fifty times more likely to appear than dominant portrayals.

The degree of harm is amplified when US nationality cues (e.g., “American”) are present in input prompts. Notably, we find that the harms we identify cannot be explained away via sycophancy, as US-centric biases persist even when replacing US nationality cues with non-US national identities in the prompts. Based on our findings, we call for further exploration of cultural harms in LLMs through methodologies that center Global Majority perspectives and challenge the uncritical adoption of US-based LLMs for the classification, surveillance, and misrepresentation of the majority of our planet.

The study employs a Question Answering (QA) model to identify nationality identity markers in narratives. By analyzing 500,000 stories generated by GPT-3.5, GPT-4, Llama 2, Claude 2.0, and PaLM 2, the research team evaluated nationality biases of these models in a U.S. context. The study also introduces a new dataset of 292,500 narratives generated by GPT4.1-Nano across 195 globally recognized nations to compare Global Majority and Minority representations.

Results show that out of 500,000 stories, only 5.4% mention a country or nationality other than the U.S., with only 1.8% involving character identity. Non-U.S. nationalities in neutral scenarios are 50 times more likely to refer to cuisine, language, or travel destinations than to a person. In power-laden stories, non-U.S. nationality characters are more likely to be portrayed as subordinate roles (98.6%) rather than dominant roles (1.6%).

This research highlights the potential biases and representational harms in LLM-generated narratives against Global Majority nationalities. By thoroughly analyzing how these models depict nationality identities in different contexts, the study provides crucial insights into understanding and mitigating cultural biases in AI systems. This is particularly significant for enterprise and government applications that rely on AI for decision-making, as these biases could impact policy-making and social equity.