Effective Filter Pruning Method Using Additional Downsampled Image for Global Pooling Applied CNN

2021 ◽  
pp. 2154029
Author(s):  
Hyunseok Kim ◽  
Bunyodbek Ibrokhimov ◽  
Sanggil Kang
Keyword(s):  
Evaluation Method ◽  
Computational Cost ◽  
Information Loss ◽  
Deep Convolutional Neural Networks ◽  
Embedded Devices ◽  
Computational Costs ◽  
Pruning Method ◽  
Original Dataset ◽  
High Resolution Images ◽  
Deep Cnn

Deep Convolutional Neural Networks (CNNs) show remarkable performance in many areas. However, most of the applications require huge computational costs and massive memory, which are hard to obtain in devices with a relatively weak performance like embedded devices. To reduce the computational cost, and meantime, to preserve the performance of the trained deep CNN, we propose a new filter pruning method using an additional dataset derived by downsampling the original dataset. Our method takes advantage of the fact that information in high-resolution images is lost in the downsampling process. Each trained convolutional filter reacts differently to this information loss. Based on this, the importance of the filter is evaluated by comparing the gradient obtained from two different resolution images. We validate the superiority of our filter evaluation method using a VGG-16 model trained on CIFAR-10 and CUB-200-2011 datasets. The pruned network with our method shows an average of 2.66% higher accuracy in the latter dataset, compared to existing pruning methods when about 75% of the parameters are removed.

scholarly journals A Survey on Deep Learning in Image Polarity Detection: Balancing Generalization Performances and Computational Costs

Electronics ◽  
2019 ◽  
Vol 8 (7) ◽  
pp. 783 ◽  
Author(s):  
Edoardo Ragusa ◽  
Erik Cambria ◽  
Rodolfo Zunino ◽  
Paolo Gastaldo
Keyword(s):  
Computational Cost ◽  
Fine Tuning ◽  
Deep Convolutional Neural Networks ◽  
Computational Costs ◽  
Advantages And Disadvantages ◽  
Complex Information ◽  
Large Sets ◽  
Learning Techniques ◽  
Feature Extractor ◽  
Overall Performance

Deep convolutional neural networks (CNNs) provide an effective tool to extract complex information from images. In the area of image polarity detection, CNNs are customarily utilized in combination with transfer learning techniques to tackle a major problem: the unavailability of large sets of labeled data. Thus, polarity predictors in general exploit a pre-trained CNN as the feature extractor that in turn feeds a classification unit. While the latter unit is trained from scratch, the pre-trained CNN is subject to fine-tuning. As a result, the specific CNN architecture employed as the feature extractor strongly affects the overall performance of the model. This paper analyses state-of-the-art literature on image polarity detection and identifies the most reliable CNN architectures. Moreover, the paper provides an experimental protocol that should allow assessing the role played by the baseline architecture in the polarity detection task. Performance is evaluated in terms of both generalization abilities and computational complexity. The latter attribute becomes critical as polarity predictors, in the era of social networks, might need to be updated within hours or even minutes. In this regard, the paper gives practical hints on the advantages and disadvantages of the examined architectures both in terms of generalization and computational cost.

scholarly journals Random Shifting for CNN: a Solution to Reduce Information Loss in Down-Sampling Layers

2017 ◽  
Author(s):  
Gangming Zhao ◽  
Jingdong Wang ◽  
Zhaoxiang Zhang
Keyword(s):  
Neural Networks ◽  
Computational Cost ◽  
Receptive Fields ◽  
Information Loss ◽  
Network Architectures ◽  
Training Process ◽  
Feature Maps ◽  
Improve Performance ◽  
Deep Convolutional Neural Networks ◽  
Random Strategy

Down-sampling is widely adopted in deep convolutional neural networks (DCNN) for reducing the number of network parameters while preserving the transformation invariance. However, it cannot utilize information effectively because it only adopts a fixed stride strategy, which may result in poor generalization ability and information loss. In this paper, we propose a novel random strategy to alleviate these problems by embedding random shifting in the down-sampling layers during the training process. Random shifting can be universally applied to diverse DCNN models to dynamically adjust receptive fields by shifting kernel centers on feature maps in different directions. Thus, it can generate more robust features in networks and further enhance the transformation invariance of down-sampling operators. In addition, random shifting cannot only be integrated in all down-sampling layers including strided convolutional layers and pooling layers, but also improve performance of DCNN with negligible additional computational cost. We evaluate our method in different tasks (e.g., image classification and segmentation) with various network architectures (i.e., AlexNet, FCN and DFN-MR). Experimental results demonstrate the effectiveness of our proposed method.

scholarly journals Performance of Adaptive Unstructured Mesh Modelling in Idealized Advection Cases over Steep Terrains

Atmosphere ◽  
2018 ◽  
Vol 9 (11) ◽  
pp. 444 ◽  
Author(s):  
Jinxi Li ◽  
Jie Zheng ◽  
Jiang Zhu ◽  
Fangxin Fang ◽  
Christopher. Pain ◽  
...  
Keyword(s):  
Unstructured Mesh ◽  
Computational Cost ◽  
Adaptive Mesh ◽  
Galerkin Finite Element Method ◽  
Adaptive Meshes ◽  
Galerkin Finite Element ◽  
Computational Costs ◽  
Cut Cell ◽  
Discontinuous Galerkin Finite Element ◽  
Terrain Following

Advection errors are common in basic terrain-following (TF) coordinates. Numerous methods, including the hybrid TF coordinate and smoothing vertical layers, have been proposed to reduce the advection errors. Advection errors are affected by the directions of velocity fields and the complexity of the terrain. In this study, an unstructured adaptive mesh together with the discontinuous Galerkin finite element method is employed to reduce advection errors over steep terrains. To test the capability of adaptive meshes, five two-dimensional (2D) idealized tests are conducted. Then, the results of adaptive meshes are compared with those of cut-cell and TF meshes. The results show that using adaptive meshes reduces the advection errors by one to two orders of magnitude compared to the cut-cell and TF meshes regardless of variations in velocity directions or terrain complexity. Furthermore, adaptive meshes can reduce the advection errors when the tracer moves tangentially along the terrain surface and allows the terrain to be represented without incurring in severe dispersion. Finally, the computational cost is analyzed. To achieve a given tagging criterion level, the adaptive mesh requires fewer nodes, smaller minimum mesh sizes, less runtime and lower proportion between the node numbers used for resolving the tracer and each wavelength than cut-cell and TF meshes, thus reducing the computational costs.

scholarly journals Accelerating Super-Resolution and Visual Task Analysis in Medical Images

2020 ◽  
Vol 10 (12) ◽  
pp. 4282
Author(s):  
Ghada Zamzmi ◽  
Sivaramakrishnan Rajaraman ◽  
Sameer Antani
Keyword(s):  
Deep Learning ◽  
High Resolution ◽  
Task Analysis ◽  
Multiple Scales ◽  
Medical Images ◽  
Computational Cost ◽  
Super Resolution ◽  
Visual Task ◽  
Learning Networks ◽  
High Resolution Images

Medical images are acquired at different resolutions based on clinical goals or available technology. In general, however, high-resolution images with fine structural details are preferred for visual task analysis. Recognizing this significance, several deep learning networks have been proposed to enhance medical images for reliable automated interpretation. These deep networks are often computationally complex and require a massive number of parameters, which restrict them to highly capable computing platforms with large memory banks. In this paper, we propose an efficient deep learning approach, called Hydra, which simultaneously reduces computational complexity and improves performance. The Hydra consists of a trunk and several computing heads. The trunk is a super-resolution model that learns the mapping from low-resolution to high-resolution images. It has a simple architecture that is trained using multiple scales at once to minimize a proposed learning-loss function. We also propose to append multiple task-specific heads to the trained Hydra trunk for simultaneous learning of multiple visual tasks in medical images. The Hydra is evaluated on publicly available chest X-ray image collections to perform image enhancement, lung segmentation, and abnormality classification. Our experimental results support our claims and demonstrate that the proposed approach can improve the performance of super-resolution and visual task analysis in medical images at a remarkably reduced computational cost.

scholarly journals CP-NAS: Child-Parent Neural Architecture Search for 1-bit CNNs

2020 ◽  
Author(s):  
Li'an Zhuo ◽  
Baochang Zhang ◽  
Hanlin Chen ◽  
Linlin Yang ◽  
Chen Chen ◽  
...  
Keyword(s):  
Computational Cost ◽  
Unified Framework ◽  
Embedded Devices ◽  
Natural Approach ◽  
Neural Architecture ◽  
Resource Limited ◽  
Training Stage ◽  
Comparable Accuracy ◽  
Kernel Level ◽  
High Computational Cost

Neural architecture search (NAS) proves to be among the best approaches for many tasks by generating an application-adaptive neural architectures, which are still challenged by high computational cost and memory consumption. At the same time, 1-bit convolutional neural networks (CNNs) with binarized weights and activations show their potential for resource-limited embedded devices. One natural approach is to use 1-bit CNNs to reduce the computation and memory cost of NAS by taking advantage of the strengths of each in a unified framework. To this end, a Child-Parent model is introduced to a differentiable NAS to search the binarized architecture(Child) under the supervision of a full-precision model (Parent). In the search stage, the Child-Parent model uses an indicator generated by the parent and child model accuracy to evaluate the performance and abandon operations with less potential. In the training stage, a kernel level CP loss is introduced to optimize the binarized network. Extensive experiments demonstrate that the proposed CP-NAS achieves a comparable accuracy with traditional NAS on both the CIFAR and ImageNet databases. It achieves an accuracy of 95.27% on CIFAR-10, 64.3% on ImageNet with binarized weights and activations, and a 30% faster search than prior arts.

Biometric Authentication Using Finger-Vein Patterns with Deep-Learning and Discriminant Correlation Analysis

2021 ◽  
pp. 2250013
Author(s):  
Aldjia Boucetta ◽  
Leila Boussaad
Keyword(s):  
Correlation Analysis ◽  
Support Vector ◽  
Svm Classifier ◽  
Identification System ◽  
Deep Convolutional Neural Networks ◽  
Finger Vein ◽  
Deep Cnn ◽  
Identification Stage ◽  
Human Finger ◽  
The Right

Finger-vein identification, a biometric technology that uses vein patterns in the human finger to identify people. In recent years, it has received increasing attention due to its tremendous advantages compared to fingerprint characteristics. Moreover, Deep-Convolutional Neural Networks (Deep-CNN) appeared to be highly successful for feature extraction in the finger-vein area, and most of the proposed works focus on new Convolutional Neural Network (CNN) models, which require huge databases for training, a solution that may be more practicable in real world applications, is to reuse pretrained Deep-CNN models. In this paper, a finger-vein identification system is proposed, which uses Squeezenet pretrained Deep-CNN model as feature extractor from the left and the right finger vein patterns. Then, combines this Deep-based features by using a feature-level Discriminant Correlation Analysis (DCA) to reduce feature dimensions and to give the most relevant features. Finally, these composite feature vectors are used as input data for a Support Vector Machine (SVM) classifier, in an identification stage. This method is tested on two widely available finger vein databases, namely SDUMLA-HMT and FV-USM. Experimental results show that the proposed finger vein identification system achieves significant high mean accuracy rates.

scholarly journals CircConv: A Structured Convolution with Low Complexity

2019 ◽  
Vol 33 ◽  
pp. 4287-4294
Author(s):  
Siyu Liao ◽  
Bo Yuan
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Deep Neural Networks ◽  
Computational Cost ◽  
Low Complexity ◽  
Deep Convolutional Neural Networks ◽  
Significant Saving ◽  
Machine Learning Applications ◽  
Fast Multiplication ◽  
Large Model

Deep neural networks (DNNs), especially deep convolutional neural networks (CNNs), have emerged as the powerful technique in various machine learning applications. However, the large model sizes of DNNs yield high demands on computation resource and weight storage, thereby limiting the practical deployment of DNNs. To overcome these limitations, this paper proposes to impose the circulant structure to the construction of convolutional layers, and hence leads to circulant convolutional layers (CircConvs) and circulant CNNs. The circulant structure and models can be either trained from scratch or re-trained from a pre-trained non-circulant model, thereby making it very flexible for different training environments. Through extensive experiments, such strong structureimposing approach is proved to be able to substantially reduce the number of parameters of convolutional layers and enable significant saving of computational cost by using fast multiplication of the circulant tensor.

Collaborative correlation filters for real-time tracking with spatial constraint

2019 ◽  
Vol 17 (03) ◽  
pp. 1950012
Author(s):  
Lifang Zhou ◽  
Hongmei Li ◽  
Weisheng Li ◽  
Bangjun Lei ◽  
Lu Wang
Keyword(s):  
Adaptive Sampling ◽  
Search Strategy ◽  
State Of The Art ◽  
Computational Cost ◽  
Background Information ◽  
Spatial Constraint ◽  
Correlation Filters ◽  
Computational Costs ◽  
Real Time Tracking ◽  
Selection Factor

Accurate scale estimation of the target plays an important role in object tracking. Most state-of-the-art methods estimate the target size by employing an exhaustive scale search. These methods can achieve high accuracy but suffer significantly from large computational cost. In this paper, we first propose an adaptive scale search strategy with the scale selection factor instead of an exhaustive scale search. This proposed strategy contributes to reducing computational costs by adaptive sampling. Furthermore, the boundary effects of correlation filters are suppressed by investigating background information so that the accuracy of the proposed tracker can be boosted. Experiments’ empirical evaluations of 61 challenging benchmark sequences demonstrate that the overall tracking performance of the proposed tracker is very successfully improved. Moreover, our method obtains the top rank in performance by outperforming 17 state-of-the-art trackers on OTB2013.

scholarly journals Evolution of Deep Convolutional Neural Networks Using Cartesian Genetic Programming

2020 ◽  
Vol 28 (1) ◽  
pp. 141-163 ◽  
Author(s):  
Masanori Suganuma ◽  
Masayuki Kobayashi ◽  
Shinichi Shirakawa ◽  
Tomoharu Nagao
Keyword(s):  
Genetic Programming ◽  
Expert Knowledge ◽  
Computational Cost ◽  
Computational Time ◽  
Network Architectures ◽  
Cartesian Genetic Programming ◽  
Deep Convolutional Neural Networks ◽  
High Performing ◽  
Network Training ◽  
Automatic Design Methods

The convolutional neural network (CNN), one of the deep learning models, has demonstrated outstanding performance in a variety of computer vision tasks. However, as the network architectures become deeper and more complex, designing CNN architectures requires more expert knowledge and trial and error. In this article, we attempt to automatically construct high-performing CNN architectures for a given task. Our method uses Cartesian genetic programming (CGP) to encode the CNN architectures, adopting highly functional modules such as a convolutional block and tensor concatenation, as the node functions in CGP. The CNN structure and connectivity, represented by the CGP, are optimized to maximize accuracy using the evolutionary algorithm. We also introduce simple techniques to accelerate the architecture search: rich initialization and early network training termination. We evaluated our method on the CIFAR-10 and CIFAR-100 datasets, achieving competitive performance with state-of-the-art models. Remarkably, our method can find competitive architectures with a reasonable computational cost compared to other automatic design methods that require considerably more computational time and machine resources.

An Evaluation of the Semi-Coupled Scheme for the Analysis of Floating Production Systems

2014 ◽  
Author(s):  
Aldo Roberto Cruces Girón ◽  
Fabrício Nogueira Corrêa ◽  
Breno Pinheiro Jacob
Keyword(s):  
Deep Water ◽  
Production Systems ◽  
Computational Cost ◽  
The Other ◽  
Coupled Analysis ◽  
Computational Costs ◽  
Analysis Techniques ◽  
Early Design Stages ◽  
Accuracy Of Results ◽  
Low Computational Cost

Analysis techniques and numerical formulations are available in a variety for mooring and riser designers. They are applied in the different stages of the design processes of floating production systems (FPS) by taking advantage of both the accuracy of results and the computational costs. In early design stages, the low computational cost is more valued with the aim of obtaining fast results and taking decisions. So in these stages it is common to use uncoupled analysis. On the other hand, in more advanced design stages, the accuracy of results is more valued, for which the use of coupled analysis is adequate. However, it can lead to excessive computing times. To overcome such high computational costs, new formulations have been proposed with the aim of obtaining results similar to a coupled analysis, but with low computational costs. One of these formulations is referred as the semi-coupled scheme (S-C). Its main characteristic is that it combines the advantages of uncoupled and coupled analysis techniques. In this way, analyses can be performed with very fast execution times and results are superior to those obtained by the classical uncoupled analysis. This work presents an evaluation of the S-C scheme. The evaluation is made by comparing their results with the results of coupled analyses. Both type of analysis were applied in a representative deep water platform. The results show that the S-C scheme have the potentially to provide results with appropriate precision with very low computational times. In this way, the S-C scheme represents an attractive procedure to be applied in early and intermediate stages of the design process of FPS.

Sign in / Sign up

Export Citation Format

Share Document