Correcting Negative Bias in Large Language Models through Negative Attention Score Alignment
Negative Attention Score Alignment (NASA) reduces negative bias in large language models, enhancing precision and recall.
Key Findings
Methodology
The study introduces Negative Attention Score (NAS) and Negative Attention Score Alignment (NASA) methods to reduce negative bias in language models by identifying and fine-tuning negatively biased attention heads. NASA is a parameter-efficient fine-tuning technique that significantly improves precision and recall without compromising generalization abilities.
Key Results
- NASA method significantly reduced the gap between precision and recall across various reasoning tasks, while maintaining or improving accuracy and F1 score. For instance, precision increased by 6% and recall by 8% on the StrategyQA dataset.
- On the GSM8K-Rephrased dataset, NASA improved recall from 33.6% to 83.8%, effectively addressing negative bias.
- Experimental results demonstrate robust performance across different prompting formats, including few-shot settings.
Significance
This research addresses the negative bias issue in binary decision tasks of large language models, enhancing reliability and reasoning capabilities. The NASA method has broad implications for academia and industry, significantly improving model performance in complex reasoning tasks.
Technical Contribution
Technical contributions include the introduction of a novel Negative Attention Score (NAS) framework for systematically probing negatively biased attention heads and parameter-efficient tuning via NASA. This method offers new theoretical guarantees and engineering possibilities compared to existing SOTA methods.
Novelty
This study is the first to systematically identify and fine-tune negatively biased attention heads, significantly reducing negative bias through the NASA method. Compared to related work, this approach provides more precise bias detection and efficient tuning strategies.
Limitations
- NASA may not completely eliminate negative bias in certain complex reasoning tasks, especially when datasets are imbalanced.
- The method relies on the quality and diversity of existing datasets, potentially performing poorly on low-quality datasets.
Future Work
Future directions include exploring NASA's application across more types of reasoning tasks and optimizing the tuning process to enhance model generalization.
AI Executive Summary
Large language models often exhibit negative bias in complex reasoning tasks, leading to high precision but low recall in binary decision-making. Existing solutions have fallen short in addressing this issue.
This paper proposes a novel Negative Attention Score Alignment (NASA) method, which reduces negative bias by identifying and fine-tuning negatively biased attention heads. NASA is a parameter-efficient fine-tuning technique that significantly improves precision and recall without compromising generalization abilities.
Experimental results show that NASA significantly reduces the gap between precision and recall across various reasoning tasks, while maintaining or improving accuracy and F1 score. This method has broad implications for academia and industry, significantly improving model performance in complex reasoning tasks.
Deep Dive
Abstract
A binary decision task, like yes-no questions or answer verification, reflects a significant real-world scenario such as where users look for confirmation about the correctness of their decisions on specific issues. In this work, we observe that language models exhibit a negative bias in the binary decisions of complex reasoning tasks. Based on our observations and the rationale about attention-based model dynamics, we propose a negative attention score (NAS) to systematically and quantitatively formulate negative bias. Based on NAS, we identify attention heads that attend to negative tokens provided in the instructions as answer candidate of binary decisions, regardless of the question in the prompt, and validate their association with the negative bias. Additionally, we propose the negative attention score alignment (NASA) method, which is a parameter-efficient fine-tuning technique to address the extracted negatively biased attention heads. Experimental results from various domains of reasoning tasks and large model search space demonstrate that NASA significantly reduces the gap between precision and recall caused by negative bias while preserving their generalization abilities.