A Verifier-Guided Explainable Reasoning Framework with Gold-Anchored QLoRA, Task-Aware Mixture-of-Experts, and Group-Relative RLVR
Improves reasoning depth to 72.20% using Gold-Anchored QLoRA and RLVR.
Key Findings
Methodology
The study proposes a verifier-guided explainable reasoning framework combining Gold-Anchored QLoRA, task-aware Mixture-of-Experts, and Group-Relative RLVR. Qwen2.5-3B-Instruct is adapted using Gold-Anchored QLoRA, followed by a lightweight router assigning logic problems to a FOL/Z3 verifier and physics problems to a formula- and unit-aware symbolic solver.
Key Results
- RLVR increases reasoning depth P3 from 50.68% to 72.20%, while hybrid P1 remains stable at 55.94%. Self-consistency improves model-only P1 from 48.86% to 50.23%, with symbolic verification providing the remaining hybrid gain.
- On 438 validation samples, RLVR primarily strengthens explicit reasoning structure, while symbolic verification complements the neural policy by improving answer reliability at the system level.
- Experiments show that five-generation self-consistency and symbolic verification contribute gains in different parts.
Significance
The study significantly enhances transparency and reasoning depth in educational QA by combining symbolic verification with neural policy. It addresses long-standing issues of reasoning inconsistency and verification difficulty, impacting both academia and industry.
Technical Contribution
The framework offers new theoretical guarantees and engineering possibilities through Gold-Anchored QLoRA and task-aware Mixture-of-Experts. Compared to existing methods, it provides significant improvements in reasoning depth and answer reliability.
Novelty
This is the first to combine Gold-Anchored QLoRA and Group-Relative RLVR for reasoning verification. Compared to related work, it fundamentally innovates in reasoning transparency and verification reliability.
Limitations
- RLVR primarily enhances reasoning depth but offers limited improvement in answer accuracy.
- Symbolic verification may not provide sufficient support in certain scenarios.
- The model may encounter errors when handling complex physical formulas.
Future Work
Future research can explore more complex verification mechanisms and broader application scenarios to further improve reasoning transparency and accuracy.
AI Executive Summary
Large language models exhibit strong reasoning capabilities, but their explanations often lack consistency or are difficult to verify. Existing solutions have not effectively addressed these issues.
This paper proposes a verifier-guided explainable reasoning framework combining Gold-Anchored QLoRA, task-aware Mixture-of-Experts, and Group-Relative RLVR. The framework uses a lightweight router to assign logic problems to a FOL/Z3 verifier and physics problems to a formula- and unit-aware symbolic solver.
Experimental results show that RLVR significantly enhances reasoning depth, while symbolic verification improves answer reliability at the system level. This method has broad application potential in educational QA, despite some limitations. Future research can further optimize verification mechanisms and expand application scenarios.
Deep Analysis
Background
In recent years, large language models have made significant strides in multi-step reasoning tasks. However, in scientific and educational QA, explainability remains a challenge. Existing methods like self-consistency and process supervision have improved reasoning to some extent but require further verification.
Core Problem
Large language models' reasoning explainability is inconsistent and difficult to verify. Correct answers may rely on invalid reasoning or inappropriate formulas, leading to answer reliability issues.
Innovation
This paper proposes a novel verifier-guided framework combining Gold-Anchored QLoRA and Group-Relative RLVR. By using a lightweight router for task-aware Mixture-of-Experts, it enhances reasoning transparency and verification reliability.
Methodology
- �� Adapt Qwen2.5-3B-Instruct using Gold-Anchored QLoRA.
- �� Use a lightweight router to assign tasks to external symbolic experts.
- �� Apply RLVR for group-relative optimization, enhancing reasoning depth.
Experiments
Experiments are conducted on the EXACT logic and physics datasets using an 80:20 split strategy. Evaluation includes the impact of five-generation self-consistency and symbolic verification on reasoning depth and answer accuracy.
Results
RLVR significantly enhances reasoning depth P3, while symbolic verification improves system-level answer reliability. Five-generation self-consistency and symbolic verification contribute gains in different parts.
Applications
This framework can be used in educational QA systems to enhance transparency and verification reliability. It can also be applied in other fields requiring high transparency and verification.
Limitations & Outlook
RLVR offers limited improvement in answer accuracy, and symbolic verification may not provide sufficient support in certain scenarios. Future research needs to further optimize verification mechanisms.
Plain Language Accessible to non-experts
Imagine a school exam system where the teacher gives questions, students answer, and then the teacher checks if the answers are correct. Our framework acts like a smart teacher, not only checking the answers but also checking if the student's thought process is reasonable. This way, even if the student's answer is correct, the teacher knows if they truly understood the problem.
ELI14 Explained like you're 14
Imagine you're playing a game with lots of puzzles. This framework is like a super helper that helps you solve puzzles and ensures you understand each step. It not only tells you the answer but also explains why doing it this way is correct. Isn't that cool?
Glossary
QLoRA (Gold-Anchored QLoRA)
An adaptation technique for large language models emphasizing authoritative answers and structured evidence.
Used to adapt Qwen2.5-3B-Instruct.
RLVR (Group-Relative RLVR)
A reinforcement learning method based on verifiable rewards, optimizing reasoning depth.
Used to enhance reasoning structure.
FOL/Z3 Verifier
A symbolic verification tool for logic problems, checking premise and conclusion consistency.
Used to verify logic problem correctness.
Symbolic Solver
A tool for physics problems, checking formula and unit consistency.
Used to verify physics problem correctness.
Self-Consistency
A method to improve model stability by generating answers multiple times.
Used to reduce single-generation variance.
Open Questions Unanswered questions from this research
- 1 How to further expand the coverage of symbolic verification to support more complex physical formulas?
- 2 How to optimize RLVR to simultaneously improve answer accuracy and reasoning depth?
Applications
Immediate Applications
Educational QA Systems
Enhance transparency and verification reliability in educational QA, suitable for online learning platforms.
Long-term Vision
Scientific Research Tool
Serve as a verification tool in scientific research, helping researchers check complex reasoning processes.
Abstract
Large language models (LLMs) show strong reasoning ability, but their explanations can remain inconsistent, weakly grounded, or difficult to verify. We propose a verifier-guided explainable reasoning framework for transparent educational question answering that combines gold-anchored QLoRA, task-aware symbolic routing, and group-relative RLVR. Qwen2.5-3B-Instruct is first adapted with field-weighted QLoRA supervision anchored to authoritative answers. A lightweight router then assigns logic problems to a FOL/Z3 verifier and physics problems to a formula- and unit aware symbolic solver. Verifier feedback is further used to support candidate evaluation, self-revision, and reward construction during RLVR. Candidate responses are evaluated along three complementary dimensions: P1 for answer correctness, P2 for evidence or unit consistency, and P3 for reasoning depth and explainability. At inference, gold-free self-consistency aggregates multiple candidate responses before an optional question-only physics verifier performs conservative system-level correction. On 438 held-out examples, RLVR increases P3 from 50.68% to 72.20%, while hybrid P1 remains approximately stable at 55.94%. Self-consistency improves model only P1 from 48.86% to 50.23%, with symbolic verification providing the remaining hybrid gain. These results indicate that RLVR primarily strengthens explicit reasoning structure, while symbolic verification complements the neural policy by improving answer reliability at the system level.