A Model of Dual Fabry-Perot Etalon-Based External-Cavity Tunable Laser Us...
Internal motion within pulsating pure-quartic soliton molecules in a fibe...
Enhanced light emission of germanium light-emitting-diode on 150 mm germa...
The Fabrication of GaN Nanostructures Using Cost-Effective Methods for Ap...
Negative-to-Positive Tunnel Magnetoresistance in van der Waals Fe3GeTe2/C...
Quantum Light Source Based on Semiconductor Quantum Dots: A Review
A High-Reliability RF MEMS Metal-Contact Switch Based on Al-Sc Alloy
Development of a Mode-Locked Fiber Laser Utilizing a Niobium Diselenide S...
Development of Multiple Fano-Resonance-Based All-Dielectric Metastructure...
Traffic Vibration Signal Analysis of DAS Fiber Optic Cables with Differen...
官方微信
友情链接

Exploring Structural Sparsity in CNN via Selective Penalty

2022-04-02

 

Author(s): Zhao, MX (Zhao, Mingxin); Peng, JB (Peng, Junbo); Yu, SM (Yu, Shuangming); Liu, LY (Liu, Liyuan); Wu, NJ (Wu, Nanjian)

Source: IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY Volume: 32 Issue: 3 Pages: 1658-1666 DOI: 10.1109/TCSVT.2021.3071532 Published: MAR 2022

Abstract: Although massive pruning methods are emerging for achieving structural sparsity in convolutional neural networks (CNN), most of them target structures such as ResNet. Meanwhile, previous works take more interest in pruning filters inside a residual block and keep the shortcut connection intact, leading to an imbalanced network structure. In this paper, we focus on the penalty-based method to prune already compact networks. In contrast to the broadly used L-1 constraint, which shrinks the parameters uniformly, we propose a novel penalty term that is similar in shape to an upside-down Laplace distribution. The penalty allows us to impose more pressure on potential weak channels but protects others during training to avoid damaging crucial channels, especially for compact architectures. We also design a candidate selection strategy to cooperate with the penalty-based training procedure. Besides, we address the residual block pruning problem by a scaling factor elimination skill, which is often ignored in other research. Our method reduces 50% parameters of MobileNet v1/v2 with a tolerable accuracy degradation. We further conduct pruning on MobileNetv1-SSDLite to compress parameters by 60%, manifesting the ability to generalize to different visual tasks. The experiment results demonstrate that our method outperforms pruning frameworks based on channel importance without a complicated tuning for hyper-parameters like search-based methods.

Accession Number: WOS:000766700400060

ISSN: 1051-8215

eISSN: 1558-2205

Full Text: https://ieeexplore.ieee.org/document/9398648



关于我们
下载视频观看
联系方式
通信地址

北京市海淀区清华东路甲35号(林大北路中段) 北京912信箱 (100083)

电话

010-82304210/010-82305052(传真)

E-mail

semi@semi.ac.cn

交通地图
版权所有 中国科学院半导体研究所

备案号:京ICP备05085259-1号 京公网安备110402500052 中国科学院半导体所声明