REFACT: Adaptive Fact Restatement for Compact and Faithful Chain-of-Thought Reasoning

The evolution of large language models (LLMs) like GPT-3 and BERT has significantly advanced NLP capabilities, especially in understanding and generating coherent text. Early efforts focused on scaling model size and training data, leading to impressive performance in tasks like translation, summarization, and question answering. However, in long-text reasoning, these models often struggle with maintaining factual consistency, path efficiency, and interpretability. Existing solutions such as retrieval-augmented generation (RAG), long-context fine-tuning, and multi-agent systems have improved contextual understanding but still face limitations in evidence utilization and path compression. Recent research emphasizes grounding reasoning in external evidence, yet most methods treat citations as static references or post-hoc annotations, lacking dynamic integration during inference. This gap motivates the development of mechanisms that enable models to adaptively select and restate evidence, ensuring more faithful and concise reasoning paths.

The core challenge in long-text reasoning is balancing evidence support with path brevity. Existing models often produce verbose, over-cited reasoning paths that rely on memorized knowledge or irrelevant information, reducing interpretability and increasing computational costs. Moreover, static citation strategies fail to adapt to varying context lengths and inference needs, leading to either insufficient evidence or excessive redundancy. This impairs the model’s ability to produce trustworthy outputs, especially in critical applications like scientific or legal reasoning where fact fidelity is paramount. Addressing this requires a mechanism for dynamic evidence selection, granularity adjustment, and path compression, which current methods do not adequately provide.

REFACT introduces a pioneering adaptive fact restatement mechanism that dynamically adjusts the granularity of cited facts based on local inference requirements. It combines multi-granularity evidence labels with a multi-objective reward system, guiding the model to cite only relevant facts at appropriate detail levels. The framework employs a teacher-student training paradigm: the teacher generates high-quality, citation-aware reasoning trajectories, which inform the student model’s learning process via supervised fine-tuning and reinforcement learning. This approach enables the model to decide when to cite, how to restate facts, and how to compress reasoning paths, significantly reducing token usage while maintaining or improving factual fidelity. The method also synthesizes diverse long-context datasets with inserted distractors, enhancing the model’s adaptability across different input lengths.

• �� First, a teacher model generates citation-aware reasoning trajectories with multi-granularity evidence tags, guiding the model to cite specific facts at different levels (entity, phrase, sentence, paragraph). • These trajectories are filtered based on four quality criteria: format correctness, factual accuracy, traceability, and answerability, ensuring high-quality training data. • The training data is constructed by inserting gold context into distractor documents of varying lengths, creating diverse long-input scenarios. • The student model is first fine-tuned with supervised learning on these data, learning to produce citation-embedded reasoning paths. • Subsequently, reinforcement learning optimizes the model’s ability to select and restate facts adaptively, guided by a reward function balancing format, accuracy, traceability, and answerability. • During inference, the model dynamically determines when to cite and how to restate facts, producing concise, faithful reasoning chains.

The evaluation employs datasets like LongBench v1/v2, LV-Eval, and ConFiQA, covering multi-hop and single-document QA across various context lengths (16k to 128k tokens). Baselines include Zero-Shot, LongAlign, LongFaith, and CARE. Metrics focus on F1, reasoning token count, evidence density, and citation accuracy. The training set comprises synthesized long contexts with inserted distractors, ensuring the model learns to handle diverse input lengths. Ablation studies confirm the importance of adaptive fact restatement and multi-granularity evidence labels. Hyperparameters such as reward weights and training epochs are tuned to balance path compactness and factual fidelity. Results demonstrate that REFACT achieves higher F1 scores, reduces reasoning tokens by approximately 30%, and improves evidence density by over 20%, especially in ultra-long contexts.

REFACT outperforms baselines with an average F1 of 64.4 on LongBench v1, with token reduction of 30%. In 128k contexts, evidence density increases by 20%, and reasoning paths are significantly shorter and more relevant. On LV-Eval, FactRecall reaches 66.3%, surpassing previous models. In ConFiQA, source fidelity improves, indicating better evidence grounding. Ablation results show that adaptive fact restatement and multi-granularity labeling are critical for these gains. Overall, the model demonstrates superior efficiency, interpretability, and factual accuracy across diverse long-text reasoning tasks.

REFACT can be directly applied to complex question answering, scientific literature analysis, and legal document reasoning, where evidence traceability and path efficiency are essential. Its ability to adaptively cite and restate facts makes it suitable for high-stakes domains requiring transparency and reliability. Long-term, integrating multi-modal data and scaling to even larger contexts could revolutionize AI-assisted research, decision-making, and knowledge management, enabling more trustworthy and efficient AI systems.

The approach depends heavily on high-quality annotated datasets and teacher models, which may limit scalability. Handling extremely long contexts (>128k tokens) still poses computational challenges. Its robustness under conflicting or noisy evidence scenarios needs further validation, especially in real-world dynamic environments. Future work should focus on improving efficiency, robustness, and multi-modal evidence integration to broaden applicability.