From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models
Political Compass probing plus controlled continued pretraining traces partisan-corpus effects into hate-speech and misinformation classifiers.
Key Findings
Methodology
The paper introduces a two-stage framework. First, it measures LM ideology with the 62-item Political Compass Test on social and economic axes. Encoders use masked-token probabilities over positive and negative stance lexicons; generators produce responses classified by a BART stance detector trained on MultiNLI. Second, RoBERTa and GPT-2 undergo controlled continued pretraining on partisan POLITICS news and Reddit corpora, then are fine-tuned on downstream datasets while initialization, task data, and hyperparameters remain fixed.
Key Results
- All 14 evaluated models exhibit measurable political leanings across the four compass quadrants. Mean absolute social-axis magnitude is 2.97 versus 0.87 economically; BERT variants are generally more socially authoritarian than GPT variants.
- Partisan data shifts ideology: RoBERTa’s social score moves from 2.97 to −3.03 after REDDIT-LEFT pretraining. Social-media corpora produce a 1.60 average social-axis change, while news produces a 0.90 average economic-axis change.
- Aggregate scores hide subgroup effects. Vanilla RoBERTa reaches 88.74/81.15 BACC/F1 on HATE-IDENTITY; REDDIT-RIGHT falls to 88.34/80.19. On MISINFORMATION, scores fall from 88.80/88.37 to 86.01/85.05.
Significance
This study connects corpus bias, model ideology, and application fairness in one empirically testable pipeline. It shows that even relatively clean news and forum data, after hateful-content filtering, can transmit polarization into high-stakes classifiers. For research, average accuracy is insufficient evidence of fairness. For industry, model cards and audits should trace sources, ideological coordinates, identity groups, and source-specific errors rather than treating a model as politically neutral by default.
Technical Contribution
The main technical contribution is an operational political-ideology probe: 62 statements are converted into social and economic scores from −10 to 10 using masked-token probability aggregation or generated-answer stance detection. The paper then performs a controlled causal-style comparison through continued pretraining, changing only partisan corpora. Finally, it evaluates downstream behavior by identity-target categories and misinformation sources, adding human validation: stance detection achieves 0.97 accuracy on clear responses with Fleiss’ kappa 0.85.
Novelty
Unlike work centered on politicians, synthetic stereotypes, or annotator disagreement, this paper traces naturally occurring media bias through the complete path from corpus to LM to socially consequential task. Its novelty is methodological integration: political-science coordinates, partisan continued-pretraining controls, and category-level fairness analysis. It does not propose a new architecture; it proposes a new measurement and experimental lens for existing LMs.
Limitations
- The evidence is mainly English-language and limited to selected architectures, news, Reddit, hate-speech, and misinformation datasets; results may not transfer across cultures, languages, or deployment settings.
- Political Compass scores, the 0.3 lexicon threshold, and the stance detector are proxies. They may reflect prompt sensitivity, lexical artifacts, or evaluator bias rather than stable internal ideology.
- Fairness is assessed mainly through category performance, not causal harm, calibration, or intersectional outcomes; the datasets also cover limited identities and media sources.
Future Work
Future work should test multilingual and multicultural settings, broader tasks, and interventions such as source balancing, data reweighting, debiased pretraining, value conditioning, and model editing. The community also needs longitudinal data audits, intersectional calibration, causal fairness measures, and systematic studies of RLHF, scale, training duration, prompts, and temporal political change.
AI Executive Summary
Language models learn more than grammar from books, news, forums, and encyclopedias: they also absorb contested social viewpoints. Feng and colleagues ask whether polarization in pretraining data survives inside models and then affects hate-speech and misinformation detectors. Earlier bias work often relied on synthetic templates or fixed stereotypes, leaving the pathway from naturally occurring media bias to downstream fairness comparatively unmeasured.
Their Political Compass framework maps 62 statements onto social and economic axes. Encoder models answer through masked-token probabilities over support and opposition lexicons; generative models answer directly, with a BART stance detector trained on MultiNLI interpreting the responses. RoBERTa and GPT-2 are then continued-pretrained on left, center, and right news or Reddit corpora from POLITICS and PushShift, before controlled fine-tuning on Yoder et al.’s hate-speech datasets and Wang’s PolitiFact misinformation data.
The results are consequential. All 14 tested LMs show political leanings; mean absolute social and economic magnitudes are 2.97 and 0.87. RoBERTa shifts from 2.97 to −3.03 socially after REDDIT-LEFT training. Aggregate task performance changes modestly, but subgroup behavior changes substantially: vanilla RoBERTa scores 88.74/81.15 BACC/F1 on HATE-IDENTITY, whereas REDDIT-RIGHT scores 88.34/80.19; misinformation scores decline from 88.80/88.37 to 86.01/85.05. The message is not that every partisan model is unusable, but that fairness requires tracing sources, ideological position, and category-level errors across the entire pipeline.
Deep Analysis
Background
Digital media has expanded political participation while intensifying debate over climate, guns, abortion, taxation, and identity. BERT, RoBERTa, and GPT-family systems learn from sources including BookCorpus, CommonCrawl, and WebText. Prior work by Blodgett et al. studied representational bias and Sap et al. examined annotation effects, but naturally occurring media ideology had rarely been tracked through to downstream fairness.
Core Problem
The precise problem is to measure an LM’s political position and determine whether that position changes predictions for socially sensitive groups or partisan media sources. A single left-right label is inadequate because social and economic ideology can diverge. Overall accuracy is also misleading when errors concentrate on particular identities or sources.
Innovation
- ��Replace a one-dimensional ideology label with Political Compass social/economic coordinates. •Unify masked-token and generation-based stance probing. •Create six comparable corpora: LEFT/CENTER/RIGHT × NEWS/REDDIT. •Use controlled continued pretraining and category-level downstream evaluation to expose bias propagation rather than only reporting aggregate scores.
Methodology
- ��Probe models with 62 statements. Encoders return the top 10 masked tokens; aggregated positive-versus-negative lexicon probability, with a 0.3 empirical threshold, maps responses to four agreement levels. Generators answer with 10 random seeds; low-confidence outputs are filtered and scored by a BART-based MultiNLI stance detector. •Continue-pretrain RoBERTa and GPT-2 on POLITICS/AllSides news and PushShift Reddit corpora, filtering potential hate with a RoBERTa TweetEval classifier. •Fine-tune RoBERTa variants on HATE-IDENTITY, HATE-DEMOGRAPHIC, and PolitiFact, reporting balanced accuracy, F1, and category breakdowns. Human validation on 110 responses gives 0.97 accuracy and κ=0.85.
Experiments
The model panel contains BERT, RoBERTa, distilBERT, ALBERT, BART, GPT-2, GPT-3, GPT-J, LLaMA, Alpaca, Codex, ChatGPT, and GPT-4 variants. Downstream datasets contain 159,872 HATE-IDENTITY, 276,872 HATE-DEMOGRAPHIC, and 29,556 MISINFORMATION instances. The main controlled comparison uses vanilla RoBERTa plus NEWS-LEFT, REDDIT-LEFT, NEWS-RIGHT, and REDDIT-RIGHT versions, with pre/post-Trump, corpus-size, and epoch analyses.
Results
Ideological direction is generally inherited from the corpus, although initial model bias is difficult to overwrite. Reddit changes social values by 1.60 on average; news changes economic values by 0.90. Post-Trump corpora move most models farther from the center. Left-leaning versions slightly outperform right-leaning versions overall. Category results show hidden variation: NEWS-LEFT reaches 89.93 on Black-targeted hate examples, while REDDIT-RIGHT reaches only 83.69 for Men; misinformation scores vary by outlet in similarly nonuniform ways.
Applications
News and social platforms can probe candidate models before deployment, report BACC/F1 by identity and source, and run regression tests after data or checkpoint changes. Model providers can publish source inventories and political-coordinate diagnostics. Auditors can require subgroup and outlet-level reporting for high-impact moderation and fact-checking systems, not merely global accuracy.
Limitations & Outlook
Political Compass is a measurement proxy, not ideological ground truth; lexicons and the stance evaluator may introduce artifacts. The study is English-centric, uses limited tasks and sources, and does not establish a complete causal account of real-world harm. More epochs and data did not push models to ±10 extremes, but RLHF, prompting, scale, domain shift, intersectionality, calibration, and deployment feedback remain insufficiently tested.
Plain Language Accessible to non-experts
Imagine a trainee journalist who has read a gigantic library. The library contains newspapers, forums, books, and encyclopedias. If one kind of writing appears more often, the trainee may absorb its habits and assumptions even if nobody explicitly tells them what to believe. The researchers first give this trainee 62 opinion questions and place the answers on a two-direction map.
Next, they let the same trainee read mostly left-leaning, right-leaning, or middle-of-the-road articles and forum posts. Everything else stays the same. If the trainee’s position moves, the reading material left a trace. That is what happens: left and right corpora generally pull the model in corresponding directions. Forums influence social issues more, while news influences economic issues more.
Finally, the trainee becomes a filter for hateful messages and false stories. Its overall report card may look similar, but it behaves differently toward different groups and media outlets. It is like a school security guard who catches roughly the same number of rule-breakers overall but searches one club much more often. The lesson is simple: a high total score does not prove fairness. We must ask what the system read, whose language it misunderstands, and whether its mistakes are unevenly distributed.
ELI14 Explained like you're 14
Picture a super-smart game NPC that has read millions of posts and articles. It learns how sentences work, but it may also pick up the vibe of the people who wrote them. Researchers gave it 62 political questions and plotted its answers on a map with two directions: social issues and money-related issues. It is not a perfect mind-reading test, but it shows that models can lean in different directions.
Then they ran a controlled experiment. The same NPC kept reading left-leaning news, right-leaning news, or different Reddit communities. No other training setting changed. After reading left-leaning Reddit, RoBERTa’s social score moved from 2.97 to −3.03. So training data is not just background wallpaper—it can nudge the model’s behavior!
Next came two missions: spot hateful language and identify misinformation. The total scores did not always change dramatically, which sounds reassuring. But wait! The model could act differently depending on the identity being targeted or the news outlet involved. A detector might look accurate on the whole while being unfair to one group, like a game anti-cheat system that accidentally flags one team more often.
That is the big takeaway. AI fairness is not just “What is the average score?” Developers should also ask: What did the model read? Which groups does it misunderstand? Does changing the training material change those mistakes? Checking these questions before launch can prevent a polished-looking system from quietly copying society’s arguments into automated decisions!
Glossary
Political Compass Test
A questionnaire that estimates social and economic ideological positions. In this paper, 62 statements become coordinates ranging from −10 to 10 on each axis.
It is the central probe for comparing original and partisan-pretrained LMs.
Mask Filling
A masked language model predicts a hidden token from its context. The paper aggregates probabilities of pro- and anti-statement lexicons to infer agreement.
Used for BERT, RoBERTa, ALBERT, and related encoder models.
Stance Detection
A classifier determines whether text supports, rejects, or is uncertain about a proposition. Here, a BART model trained on MultiNLI interprets generated LM responses.
Used for decoder and autoregressive model probing.
Continued Pretraining
Training an existing checkpoint further on new text rather than starting from random parameters. This enables a controlled test of corpus effects.
RoBERTa and GPT-2 are continued-pretrained on partisan news and Reddit data.
Balanced Accuracy
The mean recall across classes, reducing the influence of class imbalance. It is one of the paper’s principal downstream metrics.
Reported for both hate-speech and misinformation datasets.
Bias Propagation
The process by which tendencies in training data enter model behavior and create uneven downstream predictions. It links corpus composition to application-level fairness.
It is the paper’s central causal-style research narrative.
Open Questions Unanswered questions from this research
- 1 The causal connection between Political Compass scores and real-world harm remains unresolved. Future work needs interpretability, longitudinal deployment audits, and user-level outcome measures rather than proxy scores alone.
- 2 The evidence is largely English and U.S.-centered. It remains unknown whether different languages, cultures, media systems, and identity taxonomies produce the same propagation patterns.
Applications
Immediate Applications
Pre-deployment bias audit
Platforms can run the 62 statements and category-level task tests before releasing a moderation or fact-checking model. They should freeze prompts, seeds, and evaluator versions, then report BACC, F1, false-positive differences, and source-specific outcomes.
Pretraining-data registry
Model providers can document news, forum, filtering, and time-period sources, then compare political coordinates and subgroup results across checkpoints. If a source creates large disparities, teams can rebalance data, reweight examples, or add human review.
Long-term Vision
Auditable fairness supply chain
A long-term infrastructure could connect data provenance, training checkpoints, political-bias probes, and deployment outcomes. Multilingual tests, intersectional calibration, causal fairness metrics, and continuous monitoring would make high-impact content systems more traceable and correctable.
Abstract
Language models (LMs) are pretrained on diverse data sources, including news, discussion forums, books, and online encyclopedias. A significant portion of this data includes opinions and perspectives which, on one hand, celebrate democracy and diversity of ideas, and on the other hand are inherently socially biased. Our work develops new methods to (1) measure political biases in LMs trained on such corpora, along social and economic axes, and (2) measure the fairness of downstream NLP models trained on top of politically biased LMs. We focus on hate speech and misinformation detection, aiming to empirically quantify the effects of political (social, economic) biases in pretraining data on the fairness of high-stakes social-oriented tasks. Our findings reveal that pretrained LMs do have political leanings that reinforce the polarization present in pretraining corpora, propagating social biases into hate speech predictions and misinformation detectors. We discuss the implications of our findings for NLP research and propose future directions to mitigate unfairness.