Group-Shared Low-Rank Approximation for Mobile-Efficient Pointwise Convolutions in Large-Kernel CNNs

Key Findings

Methodology

This paper introduces the Channel Group-Shared (CGS) low-rank approximation leveraging Singular Value Decomposition (SVD). It decomposes pointwise convolution kernels into shared high-parameter projection matrices and channel-group-specific diagonal matrices. The approach partitions channels into groups, sharing the projection matrices across groups while learning individual diagonal scaling factors. Multiple configurations (CGS-B, CGS-S, CGS-L) balance compression and performance. Extensive experiments on RepLKNet, ConvNeXt, and SLaK demonstrate over 81% parameter reduction with less than 4.2% accuracy loss, validating the method's effectiveness for large-kernel CNNs.

Key Results

On ImageNet-1K classification, CGS-B reduces parameters by 81.3%, with only 1.8% accuracy drop, achieving 78.2% Top-1 accuracy, outperforming other compression methods.
On ADE20K segmentation and COCO detection, models maintain high performance with 85% parameter reduction and over 20% inference speedup.
The compressed models significantly lower storage and loading latency, enabling deployment on resource-constrained edge devices.

Significance

This work addresses the critical bottleneck of pointwise convolution parameters in large-kernel CNNs, enabling their practical deployment on mobile and edge devices. It bridges the gap between high-performance models and real-world constraints, pushing forward the feasibility of advanced vision models in low-resource environments. The theoretical foundation and empirical validation demonstrate a scalable, effective compression strategy that preserves accuracy while drastically reducing storage and computation demands, opening new avenues for real-time, on-device AI applications.

Technical Contribution

The core innovation is the SVD-inspired low-rank decomposition with channel group sharing, which reduces parameter count by factorizing pointwise kernels into shared projection matrices and group-specific diagonal matrices. The method introduces multiple configuration schemes to optimize the trade-off between compression ratio and accuracy. The theoretical framework guarantees geometric interpretability and efficient parameter sharing, enabling large-kernel CNNs to be compressed without significant performance loss. Extensive experiments confirm the approach's superiority over existing methods, establishing a new standard for model compression in high-capacity vision architectures.

Novelty

This is the first work to systematically incorporate SVD-based geometric structure into the compression of large-kernel CNN pointwise convolutions. Unlike prior approaches focusing solely on depthwise or standard convolutions, this method exploits the orthogonal matrix equivalence and low-rank properties to achieve substantial parameter reduction while maintaining expressiveness. The channel group-sharing paradigm introduces a new perspective on parameter efficiency, setting a foundation for future scalable model compression strategies.

Limitations

The method primarily targets the later stages of the network, where parameter redundancy is highest, and may have limited impact on early feature extraction layers.
Extreme compression ratios can lead to notable accuracy degradation, requiring careful hyperparameter tuning.
The algorithm's complexity and the need for hardware-specific optimization pose challenges for real-time deployment in some scenarios.

Future Work

Future research will explore adaptive channel grouping strategies, integrate quantization and pruning techniques, and extend the approach to multi-modal and video tasks. Additionally, optimizing the algorithm for hardware acceleration and exploring automated configuration schemes could further enhance deployment efficiency and model robustness.

AI Executive Summary

Large-kernel CNNs have revolutionized visual recognition by capturing long-range dependencies through expansive receptive fields. Architectures like RepLKNet, ConvNeXt, and SLaK utilize large convolutional kernels, significantly boosting accuracy across tasks such as image classification, detection, and segmentation. However, this performance gain comes with a steep cost: the parameter count, especially in pointwise convolutions, grows quadratically with kernel size, creating a major barrier for deployment on resource-limited edge devices.

Existing model compression efforts mainly focus on depthwise convolutions, employing low-rank approximation, weight sharing, and kernel factorization. Yet, the pointwise convolution, which accounts for over 87% of total parameters in many models, remains largely unoptimized, severely restricting the feasibility of deploying high-capacity models on smartphones and IoT devices.

This paper introduces a novel compression strategy based on Singular Value Decomposition (SVD), called Channel Group-Shared (CGS) low-rank approximation. It decomposes pointwise convolution kernels into shared high-parameter projection matrices and channel-group-specific diagonal matrices. By sharing the projection matrices across groups and learning individual scaling factors, the method achieves over 81% parameter reduction with minimal accuracy loss (<4.2%). Extensive experiments on multiple datasets demonstrate that CGS preserves model performance while drastically reducing storage, loading latency, and memory bandwidth requirements.

The significance of this work lies in bridging the gap between high-performance large-kernel CNNs and practical edge deployment. It enables pre-trained models to be compressed efficiently, facilitating real-time inference on smartphones and IoT devices. The approach opens new avenues for scalable, lightweight AI models, with potential extensions including quantization, pruning, and hardware-specific optimization.

Despite its success, the method has limitations, such as sensitivity in early layers and algorithmic complexity. Future work will focus on adaptive grouping, multi-modal extension, and hardware acceleration, aiming to further enhance the practicality and robustness of large-kernel CNNs in resource-constrained environments.

Deep Dive

Key Concepts

SVD (Singular Value Decomposition)

一种矩阵分解方法，将矩阵分解为两个正交矩阵和一个对角奇异值矩阵，用于低秩逼近和参数压缩，在本文中用于分解卷积核。

通道组共享 (Channel Group-Shared)

一种参数共享策略，将多个通道组的高参数投影矩阵共享，结合对角矩阵实现参数压缩，提升模型存储效率。

低秩逼近 (Low-Rank Approximation)

通过矩阵分解减少参数和计算量的方法，保持模型性能的同时降低复杂度，本文基于SVD实现。

大核卷积 (Large-Kernel Convolution)

使用尺寸远大于传统3×3的卷积核（如7×7到51×51）以增强感受野，提升模型性能，但参数量大。

参数共享 (Parameter Sharing)

在模型中多个部分共享相同参数，减少存储需求，本文将投影矩阵在通道组间共享以实现压缩。

Open Questions Unanswered questions from this research

1 如何在极端压缩比例下保持模型性能？未来需探索更鲁棒的参数配置和自适应机制，以应对不同任务和硬件环境的需求。

Applications

Immediate Applications

边缘设备视觉识别

支持智能手机、IoT设备上高性能视觉任务的实时处理，降低存储和计算压力，提升用户体验。

智能监控与安防

在有限硬件资源的监控摄像头中部署复杂模型，实现高精度识别与检测，保障安全。

Long-term Vision

普及移动端AI

推动高性能视觉模型在移动端普及，开启智能手机、穿戴设备的全新应用场景，未来实现无缝AI体验。

Abstract

Large-kernel Convolutional Neural Networks (CNNs) deliver remarkable performance in vision tasks by significantly expanding receptive fields, yet their quadratic parameter growth critically impedes storage-efficient edge deployment. While existing efficient architectures adopt parameter-efficient depthwise separable convolution backbones that leverage techniques like low-rank approximation and weight sharing to compress depthwise convolutions, we identify a critical oversight: pointwise convolutions dominate parameter volume (>87% in models like RepLKNet-31B) and constitute the primary deployment bottleneck on resource-constrained edge devices. This results in prohibitive storage costs and severe memory-loading constraints on resource-limited devices (e.g., smartphones with 4-12 GB Random Access Memory (RAM)). To overcome this, we propose Channel Group-Shared (CGS) low-rank approximation, a novel Singular Value Decomposition (SVD)-based parameter-sharing strategy. CGS constructs a structured low-rank paradigm isomorphic to SVD decomposition, comprising shared (high-parameter-cost) down/up-projection matrices across channel groups within a layer and channel-group-specific (low-parameter-cost) scalable diagonal matrices. This group-sharing design achieves significant parameter reduction. Extensive experiments demonstrate that large-kernel CNNs (RepLKNet, ConvNeXt, SLaK) enhanced with CGS strike an empirically favorable balance between competitive performance and substantially reduced storage costs. Crucially, by alleviating storage constraints, reducing memory bandwidth pressure during loading, and minimizing model loading latency, CGS enables the feasible deployment of pre-trained large-kernel CNN models on edge devices, thereby bridging the gap between high-performance vision models and practical edge deployment.

cs.LG