Network Pruning Using Linear Dependency Analysis on Feature Maps

2021 ◽  
Author(s):  
Hao Pan ◽  
Zhongdi Chao ◽  
Jiang Qian ◽  
Bojin Zhuang ◽  
Shaojun Wang ◽  
...  
Keyword(s):  
Feature Maps ◽  
Dependency Analysis ◽  
Network Pruning ◽  
Linear Dependency

scholarly journals Filter Pruning via Measuring Feature Map Information

Sensors ◽  
2021 ◽  
Vol 21 (19) ◽  
pp. 6601
Author(s):  
Linsong Shao ◽  
Haorui Zuo ◽  
Jianlin Zhang ◽  
Zhiyong Xu ◽  
Jinzhen Yao ◽  
...  
Keyword(s):  
Computational Cost ◽  
Limited Resources ◽  
Cross Layer ◽  
Current Layer ◽  
Feature Maps ◽  
Feature Map ◽  
Network Pruning ◽  
Pruning Method ◽  
The Relationship ◽  
Pruning Methods

Neural network pruning, an important method to reduce the computational complexity of deep models, can be well applied to devices with limited resources. However, most current methods focus on some kind of information about the filter itself to prune the network, rarely exploring the relationship between the feature maps and the filters. In this paper, two novel pruning methods are proposed. First, a new pruning method is proposed, which reflects the importance of filters by exploring the information in the feature maps. Based on the premise that the more information there is, more important the feature map is, the information entropy of feature maps is used to measure information, which is used to evaluate the importance of each filter in the current layer. Further, normalization is used to realize cross layer comparison. As a result, based on the method mentioned above, the network structure is efficiently pruned while its performance is well reserved. Second, we proposed a parallel pruning method using the combination of our pruning method above and slimming pruning method which has better results in terms of computational cost. Our methods perform better in terms of accuracy, parameters, and FLOPs compared to most advanced methods. On ImageNet, it is achieved 72.02% top1 accuracy for ResNet50 with merely 11.41 M parameters and 1.12 B FLOPs.For DenseNet40, it is obtained 94.04% accuracy with only 0.38M parameters and 110.72M FLOPs on CIFAR10, and our parallel pruning method makes the parameters and FLOPs are just 0.37M and 100.12M, respectively, with little loss of accuracy.

Modeling student knowledge with self-organizing feature maps.

1993 ◽  
Author(s):  
Steven A. Harp ◽  
Tariq Samad ◽  
Michael Villano
Keyword(s):  
Feature Maps ◽  
Student Knowledge ◽  
Self Organizing

Eliciting Relations from Requirements Text Based on Dependency Analysis

2014 ◽  
Vol 36 (1) ◽  
pp. 54-62 ◽  
Author(s):  
Tian-Ying LI ◽  
Lin LIU ◽  
De-Wang ZHAO ◽  
Yuan CAO
Keyword(s):  
Dependency Analysis

scholarly journals A Self-Spatial Adaptive Weighting Based U-Net for Image Segmentation

Electronics ◽  
2021 ◽  
Vol 10 (3) ◽  
pp. 348
Author(s):  
Choongsang Cho ◽  
Young Han Lee ◽  
Jongyoul Park ◽  
Sangkeun Lee
Keyword(s):  
Image Segmentation ◽  
Medical Image ◽  
Medical Image Segmentation ◽  
Feature Maps ◽  
Data Set ◽  
Feature Map ◽  
Adaptive Weighting ◽  
Spatially Adaptive ◽  
Wide Range ◽  
Decoder Architecture

Semantic image segmentation has a wide range of applications. When it comes to medical image segmentation, its accuracy is even more important than those of other areas because the performance gives useful information directly applicable to disease diagnosis, surgical planning, and history monitoring. The state-of-the-art models in medical image segmentation are variants of encoder-decoder architecture, which is called U-Net. To effectively reflect the spatial features in feature maps in encoder-decoder architecture, we propose a spatially adaptive weighting scheme for medical image segmentation. Specifically, the spatial feature is estimated from the feature maps, and the learned weighting parameters are obtained from the computed map, since segmentation results are predicted from the feature map through a convolutional layer. Especially in the proposed networks, the convolutional block for extracting the feature map is replaced with the widely used convolutional frameworks: VGG, ResNet, and Bottleneck Resent structures. In addition, a bilinear up-sampling method replaces the up-convolutional layer to increase the resolution of the feature map. For the performance evaluation of the proposed architecture, we used three data sets covering different medical imaging modalities. Experimental results show that the network with the proposed self-spatial adaptive weighting block based on the ResNet framework gave the highest IoU and DICE scores in the three tasks compared to other methods. In particular, the segmentation network combining the proposed self-spatially adaptive block and ResNet framework recorded the highest 3.01% and 2.89% improvements in IoU and DICE scores, respectively, in the Nerve data set. Therefore, we believe that the proposed scheme can be a useful tool for image segmentation tasks based on the encoder-decoder architecture.

scholarly journals Weight Reparametrization for Budget-Aware Network Pruning

2021 ◽  
Author(s):  
Robin Dupont ◽  
Hichem Sahbi ◽  
Guillaume Michel
Keyword(s):  
Network Pruning

scholarly journals Low-Cost Embedded System Using Convolutional Neural Networks-Based Spatiotemporal Feature Map for Real-Time Human Action Recognition

2021 ◽  
Vol 11 (11) ◽  
pp. 4940
Author(s):  
Jinsoo Kim ◽  
Jeongho Cho
Keyword(s):  
Embedded System ◽  
Real Time ◽  
Action Recognition ◽  
Processing Speed ◽  
Recognition Accuracy ◽  
Low Cost ◽  
Human Action Recognition ◽  
Human Action ◽  
Video Data ◽  
Feature Maps

The field of research related to video data has difficulty in extracting not only spatial but also temporal features and human action recognition (HAR) is a representative field of research that applies convolutional neural network (CNN) to video data. The performance for action recognition has improved, but owing to the complexity of the model, some still limitations to operation in real-time persist. Therefore, a lightweight CNN-based single-stream HAR model that can operate in real-time is proposed. The proposed model extracts spatial feature maps by applying CNN to the images that develop the video and uses the frame change rate of sequential images as time information. Spatial feature maps are weighted-averaged by frame change, transformed into spatiotemporal features, and input into multilayer perceptrons, which have a relatively lower complexity than other HAR models; thus, our method has high utility in a single embedded system connected to CCTV. The results of evaluating action recognition accuracy and data processing speed through challenging action recognition benchmark UCF-101 showed higher action recognition accuracy than the HAR model using long short-term memory with a small amount of video frames and confirmed the real-time operational possibility through fast data processing speed. In addition, the performance of the proposed weighted mean-based HAR model was verified by testing it in Jetson NANO to confirm the possibility of using it in low-cost GPU-based embedded systems.

Fast Turnaround HLS Debugging Using Dependency Analysis and Debug Overlays

2020 ◽  
Vol 13 (1) ◽  
pp. 1-26
Author(s):  
Al-Shahna Jamal ◽  
Eli Cahill ◽  
Jeffrey Goeders ◽  
Steven J. E. Wilton
Keyword(s):  
Dependency Analysis
Sign in / Sign up

Export Citation Format

Share Document