LoopQ: Quantization for Recursive Transformers

Recursive Transformer models, with their parameter reuse, offer remarkable efficiency but pose significant challenges for quantization due to their dynamic internal states. Traditional static quantization methods often fail to handle the distribution shifts and error propagation inherent in multi-round recursive computations, leading to performance degradation. To overcome this, the authors propose LoopQ, a loop-aware post-training quantization framework that introduces minimal yet effective adaptations to maintain model accuracy.

LoopQ employs a combination of techniques: loop-dependent activation scaling adjusts dynamic ranges for each recursion, while selective module fine-tuning targets modules most sensitive to quantization errors. Cross-loop state adapters align hidden states at transition points, reducing error amplification. Additionally, trajectory-aware calibration jointly optimizes all parameters along the recursive path, ensuring stability across multiple rounds.

Extensive experiments on seven benchmarks demonstrate that LoopQ significantly outperforms static PTQ baselines, achieving an average accuracy boost of 68.8% and perplexity reduction of 87.7% under W4A4 quantization. The improvements are especially pronounced in long-context tasks like LAMBADA, where recursive errors are most impactful. These results highlight the effectiveness of loop-aware strategies in enabling resource-efficient deployment of recursive models.

Overall, this work advances the state-of-the-art in model compression, providing a practical solution for deploying large recursive Transformers on edge devices. It opens avenues for further research into adaptive, hardware-aware quantization techniques, promising broader adoption of efficient deep learning models in real-world applications. Despite some limitations at ultra-low bits, LoopQ sets a new benchmark for robustness and accuracy in recursive model quantization.