Soft Filter Pruning for Accelerating Deep Convolutional Neural Networks
Soft Filter Pruning (SFP) allows pruned filters to be updated during training, maintaining model capacity and achieving over 42% FLOPs reduction with improved accuracy.
Key Findings
Methodology
SFP employs a dynamic pruning strategy where filters with low importance, measured by `2-norm, are temporarily zeroed out but remain trainable. This process involves filter importance evaluation, pruning by setting weights to zero, and a reconstruction step where pruned filters are updated during backpropagation. The approach supports training from scratch or fine-tuning pre-trained models, avoiding the performance degradation typical of hard pruning. It iteratively refines the network, preserving its expressive capacity while reducing FLOPs, and is compatible with various CNN architectures.
Key Results
- On ResNet-101, SFP achieves over 42% FLOPs reduction with a 0.2% top-5 accuracy increase on ImageNet, outperforming existing methods. For CIFAR-10, it doubles the inference speed of ResNet-110 with a 4% accuracy boost. Extensive experiments across datasets demonstrate consistent performance gains, with minimal accuracy drops at high pruning ratios. The method also supports training from scratch, yielding competitive results without reliance on pretraining.
- In ablation studies, `2-norm based importance outperforms `1-norm, especially at higher pruning rates. The approach maintains model robustness and generalization, validated across multiple architectures and datasets.
- Practical evaluations show that actual inference speedup closely matches theoretical predictions, confirming the method’s real-world efficiency.
Significance
This work addresses the critical challenge of model compression without performance loss, enabling deployment of deep CNNs on resource-constrained devices. By allowing filters to be updated post-pruning, it overcomes the limitations of traditional hard pruning, which often reduces model capacity and necessitates extensive fine-tuning. The ability to train from scratch simplifies the compression pipeline, reducing training time and complexity. The approach’s compatibility with various architectures and datasets underscores its broad applicability, promising significant impact in mobile AI, edge computing, and real-time inference scenarios.
Technical Contribution
The core innovation lies in the soft pruning mechanism that maintains trainability of pruned filters, combined with an importance evaluation based on `2-norm. This enables a non-greedy, iterative pruning process that preserves model capacity. The method integrates seamlessly into standard training routines, eliminating the need for separate fine-tuning phases. Theoretical analysis confirms the reduction in FLOPs and practical experiments validate the efficiency gains, establishing a new paradigm for structured model compression.
Novelty
This is the first work to introduce a soft, trainable pruning strategy for filters, allowing filters to be pruned and updated simultaneously during training. Unlike traditional hard pruning that permanently removes filters after pretraining, SFP dynamically adjusts filter importance, supporting from-scratch training and achieving better trade-offs between compression and accuracy. Its iterative importance-based pruning mechanism and compatibility with various architectures mark significant advances over prior static or pretraining-dependent methods.
Limitations
- While effective, the importance metric based solely on `2-norm may not fully capture filter significance in highly complex models, potentially leading to suboptimal pruning decisions. The method's performance at extremely high pruning ratios (>50%) requires further validation. Additionally, the actual speedup on hardware depends on implementation details, and the theoretical FLOPs reduction may not translate directly into real-world acceleration due to hardware and software constraints.
Future Work
Future directions include exploring multi-metric importance evaluation, integrating quantization and sparsity techniques, and extending the approach to other tasks like object detection and segmentation. Further research will focus on optimizing hardware-specific implementations to bridge the gap between theoretical and practical speedups, as well as automating hyperparameter selection for broader applicability.
AI Executive Summary
Deep convolutional neural networks (CNNs) have revolutionized computer vision, achieving remarkable accuracy across tasks. However, their increasing depth and width lead to prohibitively high computational costs, limiting deployment on resource-constrained devices like smartphones and embedded systems. Traditional model compression techniques, such as weight pruning and low-rank factorization, often rely on pretraining and result in unstructured sparsity, which is difficult to accelerate efficiently in practice. Filter pruning, a structured approach, offers a promising solution but suffers from capacity loss when filters are permanently removed.
This paper introduces Soft Filter Pruning (SFP), a novel strategy that dynamically prunes filters during training while allowing them to be updated, thus maintaining the network's expressive capacity. Unlike conventional hard pruning, which permanently discards filters, SFP uses `2-norms to evaluate filter importance, zeroing out low-importance filters but permitting their parameters to recover during backpropagation. This iterative process involves filter importance evaluation, pruning, and reconstruction, enabling the network to adaptively optimize its structure.
Experimental results on datasets like ImageNet and CIFAR-10 demonstrate the effectiveness of SFP. On ResNet-101, it reduces over 42% FLOPs with a slight accuracy increase of 0.2%, outperforming state-of-the-art methods. For CIFAR-10, it doubles inference speed with a 4% accuracy boost on ResNet-110. The approach supports training from scratch, simplifying the compression pipeline, and achieves competitive or superior performance without extensive fine-tuning.
The significance of this work lies in its ability to preserve model capacity during pruning, enabling efficient deployment without performance degradation. It offers a new paradigm for structured model compression, bridging the gap between theoretical FLOP reduction and real-world acceleration. Future work will explore multi-metric importance measures, hardware-aware optimization, and broader applications, promising to advance deep learning deployment in edge environments.
Deep Analysis
Background
近年来，深度卷积神经网络（CNN）在图像识别、目标检测等任务中取得了巨大成功，但随着模型深度和参数规模的不断扩大，计算成本和存储需求也随之增加，严重制约了其在移动端和边缘设备的应用。早期的模型压缩技术如权重剪枝、量化和低秩分解，虽然在一定程度上减轻了模型负担，但存在结构不优化、硬件加速困难等问题。滤波器剪枝作为一种结构化压缩方法，通过删除部分滤波器减少参数和计算量，已成为研究热点。代表性工作包括Li等的滤波器重要性评估、He等的LASSO筛选等，但这些方法多采用硬剪枝策略，导致模型容量大幅下降，且依赖预训练模型，流程繁琐。
Core Problem
现有滤波器剪枝方法多采用硬剪枝，即在预训练模型基础上直接删除滤波器，造成模型容量不可逆地减小，影响后续性能表现。此外，依赖预训练模型进行微调，流程复杂且耗时，限制了其在实际场景中的应用。硬剪枝还难以支持从零开始训练，影响模型的灵活性和普适性。如何在保证模型性能的前提下，提升压缩比和推理速度，成为深度模型压缩的核心难题。
Innovation
本文提出软滤波器剪枝（SFP），引入动态可更新机制，突破硬剪枝的容量限制。核心创新包括：
- �� 利用`2-范数评估滤波器重要性，筛选出低重要性滤波器；
- �� 在训练过程中逐步剪枝，允许被剪滤波器参数在反向传播中恢复，保持模型容量；
- �� 结合重建机制，使剪除的滤波器在训练中不断更新，增强模型鲁棒性；
- �� 支持从零训练或基于预训练模型剪枝，简化流程，提升效率。这些创新为模型压缩提供了新思路，兼顾性能和效率。
Methodology
- �� 训练前，计算所有滤波器的`2-范数，筛选出低重要性滤波器；
- �� 将筛选出的滤波器参数设为零，暂时剪除其贡献；
- �� 在每个训练epoch结束后，允许这些滤波器参数在反向传播中恢复，更新其值；
- �� 重复筛选、剪除、重建步骤，逐步压缩模型；
- �� 支持从零开始训练或在预训练模型基础上剪枝，避免繁琐微调。
- �� 采用统一剪枝率，简化超参数调优，提升效率。
Experiments
在CIFAR-10和ILSVRC-2012数据集上，采用ResNet系列模型验证。对比硬剪枝和微调方法，评估模型准确率、压缩比和推理速度。参数设置包括剪枝率（10%-40%）、训练轮数、学习率调整等。多次实验验证了方法的稳定性和适应性。硬件环境为GPU，测量实际推理时间，验证理论节省与实际加速的差异。结果显示，SFP在不同模型和剪枝比例下均优于传统方法，具有良好的泛化能力。
Results
在ResNet-101上，SFP实现超过42%的FLOPs剪枝，准确率反而提升0.2%；在ResNet-110上，推理速度提升两倍，准确率提升4%。在ILSVRC-2012上，剪枝后模型在Top-1和Top-5指标上均优于对比方法。消融实验表明，`2-范数筛选优于`1-范数，剪枝比例越高，性能下降越慢。实际推理时间与理论节省基本一致，验证了其实际应用潜力。
Applications
该方法适用于移动端、边缘设备等对模型大小和速度敏感的场景。只需在训练中加入SFP步骤，无需特殊硬件支持，即可实现模型压缩和加速。结合量化和稀疏技术，未来可进一步提升部署效率，推动深度模型在实际场景中的应用。
Limitations & Outlook
目前主要依赖`2-范数作为重要性指标，可能在某些复杂模型中未能充分捕捉滤波器的重要性，影响剪枝效果。极高剪枝比例可能导致性能下降，且实际加速受硬件和软件环境限制。未来需优化指标融合策略，增强鲁棒性，并验证在多任务、多架构中的泛化能力。
Plain Language Accessible to non-experts
想象你在整理一个厨房，里面有许多不同的厨具。有些厨具用得少，几乎不用，你可以把它们收起来，腾出空间给常用的厨具。但有时候，你会发现那些少用的厨具其实还能用，只要你在做饭时多用用它们。这就像模型中的滤波器，把一些用得少的滤波器“放零”，但在训练时还能重新用回来。这样，厨房既变得更整洁，又能用到所有的厨具，做饭也更快更好。这个方法就是让模型在剪掉一些滤波器的同时，还能在训练中重新“用起来”，保持模型的能力和效果。
ELI14 Explained like you're 14
想象你在学校的图书馆，有很多书。有些书很少借，几乎没人看。你可以把这些书藏起来，腾出空间给常用的书。但是，如果你只是把它们藏起来，就像扔掉一样，未来可能还会用到。于是，你决定把这些少用的书放在一边，但如果有人需要，你还可以把它们拿出来用。这就像模型中的滤波器，把一些用得少的“滤波器”设为空，但在训练时还能重新用回来。这样，图书馆既变得更整洁，又能保证所有重要的书都在，学习效果也不会变差。这个方法让模型变得更快、更小，但还能保持甚至提升性能，真是聪明又实用！
Abstract
This paper proposed a Soft Filter Pruning (SFP) method to accelerate the inference procedure of deep Convolutional Neural Networks (CNNs). Specifically, the proposed SFP enables the pruned filters to be updated when training the model after pruning. SFP has two advantages over previous works: (1) Larger model capacity. Updating previously pruned filters provides our approach with larger optimization space than fixing the filters to zero. Therefore, the network trained by our method has a larger model capacity to learn from the training data. (2) Less dependence on the pre-trained model. Large capacity enables SFP to train from scratch and prune the model simultaneously. In contrast, previous filter pruning methods should be conducted on the basis of the pre-trained model to guarantee their performance. Empirically, SFP from scratch outperforms the previous filter pruning methods. Moreover, our approach has been demonstrated effective for many advanced CNN architectures. Notably, on ILSCRC-2012, SFP reduces more than 42% FLOPs on ResNet-101 with even 0.2% top-5 accuracy improvement, which has advanced the state-of-the-art. Code is publicly available on GitHub: https://github.com/he-y/soft-filter-pruning