CForce: Boosting Parallel Decoding for dLLMs via Consistency Forcing
CForce enhances early-stage prediction reliability via self-trajectory consistency, significantly improving high-parallelism decoding in dLLMs.
Key Findings
Methodology
CForce leverages self-rollout trajectories, employing neighboring stage consistency constraints. It uses Confidence Adaptive KL (CAD) to dynamically blend forward and reverse KL divergences, enhancing early prediction accuracy. During training, predictions at different stages are aligned via stop-gradient techniques, combined with cross-entropy anchors for token stability. The approach applies to both mask-to-token (M2T) and edit-capable decodings, where later T2T refinements provide richer supervision for earlier masked states, especially in models with editing capabilities.
Key Results
- On LLaDA2.1-mini, CForce increases average Tokens Per Forward (TPF) from 6.94 to 9.08, with accuracy rising from 85.57% to 86.41%, demonstrating superior speed-quality trade-offs under high parallelism.
- In non-edit LLaDA2.0-mini, TPF improves from 3.60 to 6.42, with better few-step accuracy at fixed TPF budgets, validating robustness across scenarios.
- Theoretical analysis confirms that the consistency objective approximates minimizing early prediction errors, providing solid theoretical backing.
Significance
This work shifts focus from merely reducing decoding steps to improving early prediction reliability, crucial under aggressive parallel decoding. By enforcing model-internal consistency, it mitigates error propagation, enabling faster and more accurate generation. The method's compatibility with editing models and its theoretical guarantees make it a versatile tool for accelerating large language models in practical applications such as real-time chatbots, code assistants, and automated content creation, with broad industry implications.
Technical Contribution
The paper introduces a novel consistency distillation framework that aligns adjacent decoding stages using the model’s own trajectory, avoiding external teachers. The core innovation is the Confidence Adaptive KL mechanism, which adaptively balances forward and reverse KL divergences based on prediction confidence, ensuring stable yet mode-seeking predictions. The approach also incorporates a cross-entropy anchor for token stability, especially in edit scenarios. Theoretical bounds demonstrate that this objective effectively minimizes early prediction errors, expanding the technical frontier of model acceleration and robustness.
Novelty
This is the first work to embed neighboring stage consistency directly into large-scale dLLMs using self-sampled trajectories, rather than relying on external teachers or fixed targets. The adaptive blending of KL divergences based on confidence is a key innovation, enabling dynamic stabilization of predictions during high-parallel decoding. This approach fundamentally differs from prior distillation or calibration methods by emphasizing internal trajectory consistency, offering a new paradigm for model acceleration.
Limitations
- The effectiveness depends on the quality and diversity of pre-collected trajectories; poor trajectories may limit gains. In extremely high parallelism regimes, residual instability can occur. The theoretical analysis assumes well-trained models, which may not hold in undertrained scenarios.
Future Work
Future directions include extending the consistency framework to multi-modal tasks, integrating reinforcement learning for adaptive decoding strategies, and scaling to larger models and multi-task environments. Further research will explore more sophisticated curriculum schedules and trajectory sampling methods to enhance robustness and generalization.
AI Executive Summary
The rapid growth of large language models (LLMs) has revolutionized natural language processing, yet their deployment at high speed remains challenging. Traditional decoding methods focus on reducing the number of steps, but in high-parallel decoding scenarios, early prediction errors often propagate, degrading output quality. This bottleneck limits the practical speed gains achievable in real-world applications such as chatbots, code assistants, and automated content generation.
To address this, the paper introduces CForce, a novel training framework that enforces consistency within the model’s own decoding trajectory. By leveraging self-sampled rollout trajectories, CForce aligns predictions across neighboring stages through a carefully designed regularization mechanism. The core component, Confidence Adaptive KL (CAD), dynamically balances forward and reverse KL divergences based on the model’s confidence, ensuring stable yet flexible predictions. This approach effectively reduces early-stage errors, which are critical in high-parallelism settings.
Experimental results on the LLaDA series demonstrate the method’s efficacy. In the LLaDA2.1-mini model, CForce boosts the average Tokens Per Forward (TPF) from 6.94 to 9.08, while increasing accuracy from 85.57% to 86.41%. In non-edit models, TPF improves from 3.60 to 6.42, validating its broad applicability. The theoretical analysis confirms that the consistency objective approximates minimizing early prediction errors, providing a solid foundation for the empirical gains.
This work marks a significant step toward faster, more reliable large language models. By focusing on early prediction reliability, it opens new avenues for accelerating inference without sacrificing quality. The approach’s compatibility with editing capabilities and its theoretical guarantees suggest wide-ranging impacts across NLP tasks. Future research will explore multi-modal extensions, reinforcement learning integration, and larger-scale deployments, promising a new era of high-speed, high-accuracy AI systems.
Deep Dive
Abstract
Diffusion large language models (dLLMs) accelerate language generation by predicting multiple masks in a single forward pass. However, existing dLLMs can suffer from unreliable predictions in early denoising stages under aggressive parallelism strategies, leading to errors that can propagate to later stages. To tackle this issue, we present Consistency Forcing (CForce) for dLLMs, a distillation method to force the mask predictions of early stages to align with those of later stages. CForce trains the model on pre-collected self-rollout trajectories, thereby improving training-inference alignment. We introduce Confidence Adaptive KL Divergence as a distillation objective to conjoin the merits of forward and reverse KL. We further provide a theoretical analysis for the consistency objective to explain why CForce can approximately minimize the prediction error of early stages. Critically, the same formulation applies to both mask-to-token decoding and edit-capable decoding; in the edit-capable case, later token-to-token refinements provide additional supervision for earlier masked-state predictions. Experiments on non-edit and edit-capable LLaDA models show improved speed-quality trade-offs, especially under high-parallelism decoding budgets. Code is available at: https://github.com/inclusionAI/dFactory.