Generating Literal and Implied Subquestions to Fact-check Complex Claims

Automated fact-checking has evolved from keyword-based methods to deep learning models leveraging large pre-trained architectures like BERT, RoBERTa, and T5. Early approaches focused on explicit evidence matching, but struggled with complex claims involving implicit reasoning and background knowledge. PolitiFact and similar organizations provide valuable annotated datasets, yet lack systematic claim decomposition strategies. Recent efforts aim to improve interpretability, but most focus on highlighting relevant evidence rather than dissecting the claim into reasoning steps. The challenge remains in handling claims with implicit facets, requiring models to understand background context, logical inference, and nuanced background knowledge, which current systems inadequately address.

The core issue is that complex political claims often contain implicit information, background context, and subtle logical relations that are difficult for existing models to interpret and verify. Traditional fact-checking systems mainly rely on surface-level evidence matching, which fails to capture the deeper reasoning necessary for accurate verification. This leads to opaque decision processes and limited explainability, reducing user trust. The challenge is to develop a systematic method to decompose claims into explicit, interpretable subquestions covering both surface facts and implicit facets, enabling models to perform multi-step reasoning and provide transparent explanations. Achieving this requires annotated datasets, effective question generation models, and mechanisms to incorporate background knowledge.

This work introduces ClaimDecomp, a large-scale dataset with expert-annotated subquestions covering explicit and implicit claim facets. It also develops multi-round question generation models based on T5-3B, utilizing nucleus sampling to produce diverse, comprehensive question sets. Key innovations include: 1) systematic claim decomposition into interpretable yes-no questions; 2) focus on implicit facets like background knowledge and logical inference; 3) multi-turn question generation with background knowledge integration; 4) evaluation framework combining recall and human judgment. These contributions address the limitations of prior extractive or shallow methods, enabling more transparent, reasoning-aware fact-checking that can handle complex claims with implicit information.

• �� Data collection: From PolitiFact, gather claims and expert justification paragraphs, annotate explicit and implicit subquestions.
• �� Model training: Fine-tune T5-3B with two strategies—QG-MULTIPLE (generate all questions at once) and QG-NUCLEUS (generate questions sequentially).
• �� Training objective: Maximize recall of annotated questions, using semantic similarity for evaluation.
• �� Question generation: Apply nucleus sampling to produce diverse questions, incorporating background knowledge when available.
• �� Evaluation: Measure recall, semantic consistency, and evidence retrieval improvement. Human judges assess question relevance and clarity.
• �� Incorporate verification paragraph info in oracle settings to enhance implicit question prediction.

The dataset is split into training, validation, and test sets, with annotations from multiple annotators. Models are trained to generate question sets matching the reference count, evaluated on recall and semantic similarity. Human evaluation confirms high-quality question generation, especially for explicit questions. Ablation studies compare single-step versus multi-turn generation, background knowledge inclusion, and sampling strategies. Results show the multi-round models with background info outperform baselines, achieving 58% recall overall, with significant gains in explicit questions. The experiments demonstrate the approach’s effectiveness in decomposing complex claims and improving evidence retrieval.

The models generate 58% of the full set of subquestions, with over 74% recall for literal questions and 18% for implied ones. Human assessments rate the questions as highly relevant and clear. Incorporating background knowledge and multi-turn generation significantly improves the prediction of implicit questions. Evidence retrieval experiments show a 20% increase in relevant paragraph matching when using generated subquestions. The approach effectively captures diverse facets of complex claims, enabling more transparent and accurate fact-checking. Ablation results highlight the importance of background knowledge and multi-step reasoning for comprehensive question generation.

This methodology can be integrated into automated fact-checking platforms for political, legal, and medical claims, providing transparent reasoning paths. It aids journalists, fact-checkers, and the public by systematically dissecting claims into manageable, interpretable questions, reducing manual effort and increasing trust. The approach also supports AI systems in domains requiring nuanced understanding, such as legal document analysis or medical diagnosis verification. Long-term, combining this framework with knowledge graphs and reasoning modules could lead to fully autonomous, explainable verification systems capable of handling complex, implicit information in real-time scenarios.

Current models struggle with deeply implicit questions requiring extensive background knowledge, leading to incomplete decomposition. The dataset is domain-specific, limiting cross-domain generalization. Computational costs are high for training and inference, hindering real-time deployment. The models depend heavily on the quality of annotated data; errors in annotations can propagate. Future work should focus on integrating external knowledge sources, improving reasoning capabilities, and reducing computational overhead to enhance robustness and scalability.