HAMP-LIC: Hessian-Aware Mixed-Precision Post-Training Quantization for Learned Image Compression
HAMP-LIC introduces Hessian-based mixed-precision post-training quantization, achieving up to 4.85× compression with only 0.59% BD-rate loss.
Key Findings
Methodology
The proposed HAMP-LIC framework employs a four-stage strategy: first estimating layer sensitivity via Hessian trace; second refining sensitivity with task-aware metrics; third allocating bit-widths under a global size constraint using optimization; finally applying block-wise reconstruction with calibration set refinement. This approach integrates second-order information with rate-distortion objectives, dynamically adjusting bit-widths to enhance compression efficiency while ensuring cross-platform numerical consistency.
Key Results
- On Minnen2018 and Cheng2020 models, HAMP-LIC achieves up to 4.85× model compression with only 0.59% BD-rate increase, outperforming existing fixed and mixed-precision PTQ methods.
- Across datasets like Kodak and CLIC, the compressed models maintain near-original image quality and eliminate encoding-decoding mismatches across heterogeneous hardware.
- Sensitivity-guided bit allocation and block reconstruction optimization significantly reduce quantization errors, improving reconstruction fidelity at ultra-low bit-widths.
Significance
This work addresses the critical challenge of deploying high-performance LIC models on resource-constrained devices by enabling ultra-low-bit quantization without performance loss or cross-platform inconsistencies. It advances the theoretical understanding of second-order sensitivity in neural networks and offers practical solutions for real-world applications such as mobile image sharing and edge computing, bridging the gap between research and industry deployment.
Technical Contribution
The key innovations include leveraging Hessian trace to quantify layer sensitivity, integrating rate-distortion metrics for task-aware adjustment, formulating bit-width allocation as a Pareto-front constrained optimization, and employing block-level reconstruction with adaptive scaling. These contributions collectively enable efficient, robust, and theoretically grounded mixed-precision quantization tailored for LIC models, surpassing prior heuristic or gradient-based methods.
Novelty
This is the first study to incorporate Hessian second-order information into LIC model quantization, specifically designing a task-aware sensitivity refinement and a Pareto-based global optimization framework. Unlike previous works that relied on heuristic or uniform approaches, this method dynamically allocates bits based on geometric and task-specific metrics, achieving superior compression and robustness.
Limitations
- Hessian trace estimation, while efficient, still incurs computational overhead in very deep networks, limiting scalability. The method's effectiveness depends on the representativeness of calibration data, which may not generalize well to all deployment scenarios.
- The current framework primarily targets static image compression; extending to video or real-time streaming involves additional challenges such as temporal consistency.
- Further hardware-aware optimization and integration with pruning or neural architecture search could enhance practical deployment.
Future Work
Future research will explore adaptive sensitivity estimation methods for ultra-deep networks, incorporate hardware-aware constraints, and extend the framework to video compression. Combining with neural architecture search and pruning techniques could further improve efficiency and robustness, facilitating broader industrial adoption.
AI Executive Summary
Deep learning-based learned image compression (LIC) models have revolutionized image coding by surpassing traditional codecs in rate-distortion performance. However, their high computational complexity and sensitivity to numerical precision hinder real-world deployment, especially on resource-limited devices. Uniform fixed-precision quantization, while simple, often causes severe quality degradation at low bit-widths and introduces cross-platform inconsistencies. Addressing these challenges, this paper presents HAMP-LIC, a Hessian-aware mixed-precision post-training quantization framework that intelligently allocates bits across model blocks.
The core innovation lies in estimating block-wise sensitivity using the Hessian trace, capturing second-order importance, and refining these estimates with task-aware rate-distortion metrics. This sensitivity profile guides a global constrained optimization, employing Pareto-frontier search to assign optimal bit-widths that balance compression ratio and reconstruction quality. Block-wise reconstruction and calibration further suppress quantization errors, ensuring stable performance across heterogeneous hardware.
Extensive experiments on models like Minnen2018 and Cheng2020 demonstrate that HAMP-LIC achieves up to 4.85× compression with only 0.59% BD-rate loss, outperforming existing PTQ methods. It also effectively eliminates cross-platform encoding-decoding mismatches, a critical requirement for practical deployment. The framework's ability to combine geometric second-order information with task-specific objectives marks a significant step forward in LIC model compression.
This work not only enhances the efficiency and robustness of learned image compression but also provides a generalizable approach for neural network quantization. Its implications extend to mobile devices, edge computing, and real-time streaming, where high compression efficiency and numerical stability are paramount. Future directions include extending the methodology to video compression, integrating hardware-aware constraints, and exploring adaptive sensitivity estimation for ultra-deep networks.
Deep Analysis
Background
近年来，深度学习在图像压缩中的应用不断突破传统编码技术，代表性模型如Minnen2018和Cheng2020通过非线性变换和概率模型实现了优异的压缩性能。然而，这些模型普遍存在参数量大、计算复杂度高的问题，限制了其在移动端和边缘设备的部署。模型量化作为减小模型体积和提升推理速度的重要手段，逐渐成为研究热点。早期方法多采用均匀量化，虽简便但在低比特宽度下性能迅速下降。近年来，混合精度量化结合二阶信息和任务导向指标，为模型压缩提供了更细粒度的调控策略。LIC模型的特殊结构如超先验（hyperprior）和分析-合成变换，进一步增加了敏感性估算和比特分配的难度。综上，如何在保证模型性能的同时实现极端压缩，成为当前研究的核心难题。
Core Problem
尽管LIC模型在压缩效率上表现优异，但其在低比特宽度部署中面临两个主要难题：一是硬件异构环境导致的数值不一致，造成编码解码不匹配；二是模型参数的敏感性差异使得均匀量化导致性能大幅下降。传统方法难以兼顾压缩比和重建质量，且缺乏有效的敏感性评估机制，限制了低比特压缩的实用性。如何精准识别不同层或块的敏感性，合理分配比特宽度，成为提升LIC模型低比特表现的关键。
Innovation
本研究的核心创新在于引入Hessian迹作为敏感性指标，结合任务导向的率失真指标，提出多层次比特宽度分配策略。具体包括：• 利用Hessian-vector乘积高效估算每层的二阶敏感性，捕捉模型对参数扰动的局部几何特性；•设计任务导向的敏感性调整机制，将率失真指标融入敏感性评估，使比特分配更贴合实际压缩目标；•提出基于Pareto前沿的全局优化算法，极大降低搜索复杂度，实现多块块级比特宽度的联合优化；•采用块级重建和校准集优化，减缓量化误差，确保模型在不同平台上的数值一致性。该方法在理论和工程上均优于传统的均匀或启发式策略。
Methodology
- �� 首先，利用Hessian-vector乘积技术，结合Hutchinson方法，快速估算每个块的Hessian迹，获得几何敏感性指标；• 其次，结合任务导向的率失真指标，调整敏感性值，构建更符合实际压缩目标的敏感性列表；• 之后，将比特宽度分配问题转化为带约束的全局优化，采用Pareto前沿搜索策略，减少搜索空间，得到最优比特配置；• 最后，通过块级重建和校准集优化，调整量化参数，最小化量化误差，提升重建质量。整个流程实现了敏感性评估、比特调度和误差控制的闭环优化。
Experiments
在Kodak、CLIC等公开数据集上，采用Minnen2018和Cheng2020模型作为基准，比较了HAMP-LIC与传统均匀和启发式混合精度方法的性能。指标包括BD率、PSNR、模型压缩比和跨平台一致性。超参数如比特宽度集和模型预算在实验中调优。通过消融实验验证敏感性估算、任务导向调整和优化策略的贡献。结果显示，HAMP-LIC在压缩比上提升至4.85倍，BD率仅增加0.59%，且在不同硬件平台上实现了无编码解码差异。
Results
实验表明，HAMP-LIC在多个场景中均优于对比方法，压缩比提升显著，BD率损失极小。具体而言，在Cheng2020模型上，采用6比特量化时，BD率仅增加0.59%，而模型压缩达4.85倍；在Kodak和CLIC数据集上，重建质量几乎无差异，验证了跨平台数值一致性。消融分析显示，敏感性估算和任务导向调整是性能提升的关键因素，优化算法有效降低了搜索复杂度，保证了算法的实用性。
Applications
该方法适用于需要在硬件资源有限环境中部署高性能LIC模型的场景，如移动设备、边缘计算和实时视频传输。通过极端压缩，显著降低存储和计算成本，满足实际应用对低延迟和高效率的需求。未来，结合硬件感知和模型剪枝，有望实现更广泛的工业化部署，推动深度学习在图像压缩领域的普及。
Limitations & Outlook
目前，Hessian迹估算在极深网络中仍存在一定计算成本，且对校准数据的代表性敏感。此外，算法主要针对静态图像压缩；在视频场景中动态变化的适应性和实时性仍需优化。未来应在降低计算复杂度、增强鲁棒性和扩展到多模态任务方面持续努力。
Plain Language Accessible to non-experts
想象你在整理一个大型仓库，每个货架上堆满了不同的商品。为了节省空间，你决定用不同的箱子装不同的商品：一些商品很重要，不能压缩太多，否则会损坏；一些商品不那么重要，可以用更小的箱子装。你还需要确保不同仓库之间的商品摆放一致，否则会出现混乱。这个过程就像在训练一个智能系统，让它学会根据商品的重要性（敏感性）合理分配空间（比特宽度），同时保证仓库的整体容量（模型大小）不超标。这个系统会不断调整每个货架的箱子大小，确保既节省空间，又保证商品完好无损，最终实现高效、可靠的仓库管理。
ELI14 Explained like you're 14
想象你在玩一个超级复杂的拼图游戏，你有很多块拼图要拼在一起。每一块拼图都代表一个部分的图片，有些部分很重要，不能拼错；有些部分可以拼得更小一点。你的任务是用最少的空间把拼图拼好，但又不能让图片变得模糊或错乱。为了做到这一点，你会先用一种聪明的方法，估算每一块拼图的重要程度，就像用放大镜观察每块的细节。然后，根据每块的重要性，决定用大一点的盒子装重要的拼图，用小一点的盒子装不那么重要的。最后，你还会用一种特别的方法，把拼图重新调整，确保每一块都拼得又快又准。这样，你就能用更少的空间，拼出一幅清晰的图片，而且还能保证不同的朋友用不同的设备都能看得一样清楚，不会出现颜色或细节不一致的问题。
Abstract
Use this plain-text version for the arXiv abstract field: Learned image compression (LIC) models achieve strong rate-distortion performance but are hindered by high computational complexity and encoding-decoding mismatches across heterogeneous hardware platforms. Uniform fixed-precision quantization alleviates these issues but suffers severe quality degradation at low bit widths because it ignores differences in the quantization sensitivities of individual layers. To enable efficient and accurate low-bit deployment of pretrained LIC models, we propose HAMP-LIC, a Hessian-aware mixed-precision post-training quantization (PTQ) framework with a four-stage optimization strategy. First, block-wise sensitivity is estimated from the Hessian trace to capture second-order importance. Second, a task-aware refinement module adjusts these sensitivities by jointly considering quantization distortion and rate-distortion performance. Third, guided by the refined sensitivity profile, bit widths are allocated under a global model-size constraint to balance efficiency and reconstruction quality. Finally, block-wise reconstruction using a small calibration set further suppresses quantization error. Experiments on representative LIC models, including Minnen2018 and Cheng2020, demonstrate that HAMP-LIC achieves up to 4.85x model compression with as little as 0.59% BD-rate loss. It consistently outperforms existing fixed- and mixed-precision PTQ methods across multiple datasets while completely eliminating cross-platform encoding-decoding errors.