Investigating Multi-Hop Factual Shortcuts in Knowledge Editing of Large Language Models
Erasing shortcut neurons significantly reduces multi-hop knowledge editing failures and improves reasoning reliability.
Key Findings
Methodology
The study analyzes Knowledge Neurons to uncover multi-hop factual shortcuts and proposes a neuron-erasing method to mitigate their impact on knowledge editing. Experiments use the MQUAKE-CF-3K dataset and pretraining corpora like Dolma and Wikipedia for frequency analysis.
Key Results
- Result 1: Shortcut strength correlates strongly with co-occurrence frequency of initial and terminal entities in pretraining corpora; instances with >10 occurrences are more prone to shortcuts.
- Result 2: Few-shot prompting relies more on shortcuts compared to Chain-of-Thought prompting, which better guides step-by-step reasoning.
- Result 3: Erasing shortcut-related neurons significantly reduced multi-hop editing failures, e.g., GPT-J failure rate dropped from 25.83% to 22.13%.
Significance
This study reveals the reliance of LLMs on factual shortcuts in multi-hop reasoning and proposes mitigation strategies. It enhances the reliability of knowledge editing and opens new avenues for optimizing reasoning capabilities.
Technical Contribution
Introduced a neuron-erasing method to mitigate shortcuts, combining frequency analysis and neuron localization techniques. Demonstrated strong correlation between shortcuts and co-occurrence frequency in pretraining corpora.
Novelty
First systematic analysis of multi-hop factual shortcuts' impact on knowledge editing, proposing a neuron-erasing solution that addresses consistency issues in multi-hop reasoning.
Limitations
- Limitation 1: Erasing neurons may affect reasoning on other related knowledge.
- Limitation 2: Method relies on co-occurrence frequency, leaving low-frequency shortcuts unresolved.
- Limitation 3: Experiments limited to GPT-J and LLaMA-2, lacking broader model coverage.
Future Work
Future work could explore finer neuron localization methods, reduce shortcut formation during pretraining, and extend to more models and task scenarios.
AI Executive Summary
Large language models (LLMs) excel in knowledge recall and reasoning, yet their reliability in multi-hop reasoning remains underexplored. This study uncovers the phenomenon of factual shortcuts in multi-hop reasoning, originating from direct associations between initial and terminal entities in pretraining corpora.
By analyzing Knowledge Neurons, the study finds that shortcut strength correlates with entity co-occurrence frequency, and Few-shot prompting triggers shortcuts more often than Chain-of-Thought prompting. Experiments reveal that approximately 20% of failures in multi-hop knowledge editing stem from shortcuts, particularly in high-frequency co-occurrence cases.
To mitigate these risks, the study proposes erasing shortcut-related neurons, significantly reducing multi-hop editing failures. This approach not only improves reasoning consistency but also offers new directions for enhancing multi-hop reasoning capabilities.
Deep Dive
Abstract
Recent work has showcased the powerful capability of large language models (LLMs) in recalling knowledge and reasoning. However, the reliability of LLMs in combining these two capabilities into reasoning through multi-hop facts has not been widely explored. This paper systematically investigates the possibilities for LLMs to utilize shortcuts based on direct connections between the initial and terminal entities of multi-hop knowledge. We first explore the existence of factual shortcuts through Knowledge Neurons, revealing that: (i) the strength of factual shortcuts is highly correlated with the frequency of co-occurrence of initial and terminal entities in the pre-training corpora; (ii) few-shot prompting leverage more shortcuts in answering multi-hop questions compared to chain-of-thought prompting. Then, we analyze the risks posed by factual shortcuts from the perspective of multi-hop knowledge editing. Analysis shows that approximately 20% of the failures are attributed to shortcuts, and the initial and terminal entities in these failure instances usually have higher co-occurrences in the pre-training corpus. Finally, we propose erasing shortcut neurons to mitigate the associated risks and find that this approach significantly reduces failures in multiple-hop knowledge editing caused by shortcuts.