Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection
INLP (Iterative Nullspace Projection) removes bias from neural representations by iterative training of linear classifiers and nullspace projection, significantly reducing gender bias in word embeddings.
Key Findings
Methodology
The proposed INLP method iteratively trains linear classifiers to predict a protected attribute (e.g., gender, race). After each classifier is trained, its parameter matrix is used to compute a nullspace projection matrix, which projects the representations onto the subspace orthogonal to the learned bias directions. This process is repeated multiple times, each time removing additional bias directions identified by the classifiers. The core mechanism involves: training classifiers, calculating nullspace basis, deriving projection matrices, and applying these projections to the data representations. The iterative process ensures that the remaining representations are linearly guarded against the protected attribute, effectively neutralizing linear bias signals while preserving as much of the original information as possible.
Key Results
- In word embedding debiasing, INLP reduced the linear gender classification accuracy from 100% to 49.3%, outperforming manual direction removal which only lowered it to 80.7%. Nonlinear classifiers like MLP still achieved 85% accuracy, indicating residual nonlinear gender information. Visualization with t-SNE showed the gender-biased clusters became indistinguishable after INLP, with overlap dropping from 83.88% to 0.44%. WEAT tests confirmed significant reduction in stereotypical associations. In fairness classification, INLP decreased TPR-GAP from 0.76 to 0.16, demonstrating improved fairness.
- The results demonstrate that INLP effectively removes linear bias signals from word embeddings and improves fairness in classification tasks. The method's ability to identify and neutralize multiple bias directions iteratively ensures comprehensive bias mitigation, surpassing prior approaches that relied on predefined bias directions or adversarial training. The experimental data validate the theoretical guarantees of the method, showing its robustness and practical utility.
Significance
This research addresses a critical challenge in NLP: mitigating embedded biases that can lead to unfair or discriminatory outcomes. By providing a mathematically grounded, scalable, and data-driven approach, INLP advances the state-of-the-art in bias control. Its capacity to systematically identify and remove multiple bias directions without prior assumptions enhances fairness in NLP applications, such as word embeddings and classifier decision-making. This work bridges the gap between theoretical bias understanding and practical bias mitigation, paving the way for more equitable AI systems. Its implications extend beyond NLP, offering a general framework for bias control in high-dimensional representations across machine learning domains.
Technical Contribution
The main technical innovation is the development of the INLP algorithm, which combines iterative training of linear classifiers with nullspace projection to systematically eliminate bias directions. Unlike prior methods that rely on manually specified bias directions or adversarial training, INLP learns the bias directions directly from data, ensuring comprehensive coverage. The algorithm guarantees that after multiple iterations, the data representations are projected onto the intersection of the nullspaces of all learned classifiers, making the protected attribute linearly uninformative. Theoretical analysis confirms that the resulting projection matrix is valid and minimally disruptive to the data structure, preserving as much useful information as possible while removing bias signals. This approach introduces a new paradigm for bias control that is both mathematically rigorous and practically effective.
Novelty
INLP's novelty lies in its iterative learning and projection process, which automatically discovers multiple bias directions rather than relying on a single predefined axis. This iterative approach ensures a more thorough removal of bias signals embedded in high-dimensional spaces, capturing subtle and complex bias patterns that manual methods might miss. Compared to existing projection-based debiasing techniques, INLP offers a systematic, data-driven, and scalable solution that adapts to the actual bias structure in the representations. Its theoretical guarantees and empirical performance set it apart as a significant advancement in bias mitigation methods.
Limitations
- INLP primarily targets linear bias signals; nonlinear biases may persist, requiring more sophisticated methods for complete bias removal.
- Multiple iterations can lead to over-projection, potentially degrading the semantic quality of the representations, especially in high-dimensional spaces.
- Computational cost increases with the number of iterations and data size, which may limit scalability in large-scale applications.
- The effectiveness depends on the quality of the trained classifiers; if classifiers fail to capture certain bias directions, residual bias may remain.
- The method assumes that bias signals are linearly separable, which may not hold for all types of biases or complex societal stereotypes.
Future Work
Future research could explore extending INLP to handle nonlinear biases, possibly integrating kernel methods or deep learning-based bias detection. Improving computational efficiency, perhaps through incremental or approximate algorithms, would enhance scalability. Combining INLP with other fairness techniques, such as causal inference or counterfactual fairness, could address residual nonlinear biases. Additionally, developing automated bias detection and quantification tools would facilitate broader adoption. Extending this framework to multimodal data, such as images and text, and applying it to real-world deployment scenarios, remains an important direction to ensure AI fairness across diverse applications.
AI Executive Summary
Bias in neural representations, especially in NLP models, poses significant ethical and practical challenges. Word embeddings like GloVe and BERT often encode societal stereotypes, such as gender biases, which can influence downstream tasks like sentiment analysis, machine translation, and hiring classifiers. Existing mitigation strategies, including predefined bias directions and adversarial training, have limitations: they either depend on human intuition to define bias axes or involve complex, unstable training procedures. These approaches often fail to fully eliminate biases, especially subtle or high-dimensional ones.
To address these issues, this paper introduces INLP (Iterative Nullspace Projection), a novel algorithm that systematically removes linear bias signals from neural representations through an iterative process. The core idea is to train a sequence of linear classifiers, each identifying a bias direction in the data. Once trained, the classifier's parameter matrix is used to compute a nullspace projection matrix, which projects the data onto a subspace orthogonal to that bias direction. This process is repeated multiple times, each time training a new classifier on the already projected data, progressively removing additional bias directions. The mathematical foundation guarantees that after multiple iterations, the data is projected onto the intersection of all learned nullspaces, effectively neutralizing the linear predictability of the protected attribute.
Experimental results demonstrate the effectiveness of INLP in multiple contexts. In word embedding debiasing, the method reduces the ability of linear classifiers to predict gender from 100% accuracy to below 50%, with nonlinear classifiers still achieving high accuracy, indicating that nonlinear biases may persist. Visualization with t-SNE shows that gender-biased clusters become indistinguishable after INLP, and WEAT tests confirm a significant reduction in stereotypical associations. In fairness classification tasks, INLP reduces the TPR-GAP (True Positive Rate gap) from 0.76 to 0.16, indicating a substantial improvement in fairness metrics.
These findings highlight the potential of INLP as a scalable, mathematically grounded tool for bias mitigation. Its ability to identify and remove multiple bias directions iteratively makes it more comprehensive than prior methods. The approach is particularly valuable for applications where fairness and transparency are critical, such as hiring systems, loan approval algorithms, and social media content moderation.
Looking ahead, future work could extend INLP to nonlinear bias signals, improve computational efficiency, and integrate with causal fairness frameworks. The development of automated bias detection tools and application to multimodal data could further enhance its impact. Overall, INLP offers a promising pathway toward more equitable AI systems, balancing technical rigor with practical applicability, and addressing a pressing societal need for fairer machine learning models.
Deep Dive
Abstract
The ability to control for the kinds of information encoded in neural representation has a variety of use cases, especially in light of the challenge of interpreting these models. We present Iterative Null-space Projection (INLP), a novel method for removing information from neural representations. Our method is based on repeated training of linear classifiers that predict a certain property we aim to remove, followed by projection of the representations on their null-space. By doing so, the classifiers become oblivious to that target property, making it hard to linearly separate the data according to it. While applicable for multiple uses, we evaluate our method on bias and fairness use-cases, and show that our method is able to mitigate bias in word embeddings, as well as to increase fairness in a setting of multi-class classification.
References (20)
Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
Tolga Bolukbasi, Kai-Wei Chang, James Y. Zou et al.
Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them
Hila Gonen, Yoav Goldberg
Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting
Maria De-Arteaga, Alexey Romanov, Hanna M. Wallach et al.
Assessing BERT's Syntactic Abilities
Yoav Goldberg
Adversarial Removal of Demographic Attributes from Text Data
Yanai Elazar, Yoav Goldberg
Demographic Dialectal Variation in Social Media: A Case Study of African-American English
Su Lin Blodgett, L. Green, Brendan T. O'Connor
Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies
Tal Linzen, Emmanuel Dupoux, Yoav Goldberg
Bag of Tricks for Efficient Text Classification
Armand Joulin, Edouard Grave, Piotr Bojanowski et al.
Unsupervised Domain Adaptation by Backpropagation
Yaroslav Ganin, V. Lempitsky
SimLex-999: Evaluating Semantic Models With (Genuine) Similarity Estimation
Felix Hill, Roi Reichart, A. Korhonen
Large-scale learning of word relatedness with constraints
Guy Halawi, G. Dror, E. Gabrilovich et al.
Representation Learning: A Review and New Perspectives
Yoshua Bengio, Aaron C. Courville, P. Vincent
Scikit-learn: Machine Learning in Python
Fabian Pedregosa, G. Varoquaux, Alexandre Gramfort et al.
A Study on Similarity and Relatedness Using Distributional and WordNet-based Approaches
Eneko Agirre, Enrique Alfonseca, Keith B. Hall et al.
Support vector machines
Ingo Steinwart, A. Christmann
V-Measure: A Conditional Entropy-Based External Cluster Evaluation Measure
A. Rosenberg, Julia Hirschberg
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee et al.
GENERATIVE ADVERSARIAL NETS
Individualized Treat, Jinsung Yoon
Projectors on intersections of subspaces
Adi Ben-Israel
Visualizing Data using t-SNE
L. Maaten, Geoffrey E. Hinton
Cited By (20)
Where Do Multilingual Vision-Language Encoders Fail on Low-Resource Languages?
How Output Format Confounds Data Quality and Capability in Instruction Tuning
MANCE: Manifold Aware Concept Erasure
Frozen Brain-MRI Foundation Models Are Site Fingerprints
Causal Interventions on Continuous Variables: A Case Study on Verb Bias in Steering Vectors for In-Context Learning
Debiasing Without Protected Attributes: Latent Concept Erasure from Textual Profiles
Refusal Beyond a Single Direction: A Preliminary Comparison of Diff-in-Means and INLP
Mitigating Label Bias with Interpretable Rubric Embeddings
STRIDE: Training Data Attribution via Sparse Recovery from Subset Perturbations
Vectors Are Not Neutral: Sensitive-Information Inference from Exported LLM Representations in Summarization
Abduction-Deduction Entanglement: Domain Generalization via Representation Transplants
Causal Tongue-Tie: LLMs Can Encode Causal Direction, But Their Yes/No Outputs Fail to Express
Measuring Alignment-Induced Activation Shifts Correctly: A Template-Controlled Difference-in-Differences Protocol
Early Warning Signals for OpenVLA Failure under Visual Distribution Shift
A Geometric Perspective on Composable Emotion Steering in Text-to-Speech Models
Controlling Tool Use with Heading-Specific Activation Steering
Resist and Update: Counterfactual Report Coordinates for Incentive-Compatible LLMs
Certified Domain Consistency for Multi-Domain Retrieval: Label-Free Per-Domain Contamination Control with Conformal Risk Guarantees
Toward Localizing and Repairing Bias in Transformer Attention Heads
Unbiased Open World Regularization for Fair Self-Supervised Learning