scholarly journals COP: Customized Deep Model Compression via Regularized Correlation-Based Filter-Level Pruning

2019 ◽  
Author(s):  
Wenxiao Wang ◽  
Cong Fu ◽  
Jishun Guo ◽  
Deng Cai ◽  
Xiaofei He
Keyword(s):  
Neural Network ◽  
State Of The Art ◽  
Computational Cost ◽  
Cross Layer ◽  
Optimal Solutions ◽  
Deep Convolutional Neural Networks ◽  
Model Compression ◽  
Deep Model ◽  
Network Compression ◽  
Novel Algorithm

Neural network compression empowers the effective yet unwieldy deep convolutional neural networks (CNN) to be deployed in resource-constrained scenarios. Most state-of-the-art approaches prune the model in filter-level according to the "importance" of filters. Despite their success, we notice they suffer from at least two of the following problems: 1) The redundancy among filters is not considered because the importance is evaluated independently. 2) Cross-layer filter comparison is unachievable since the importance is defined locally within each layer. Consequently, we must manually specify layer-wise pruning ratios. 3) They are prone to generate sub-optimal solutions because they neglect the inequality between reducing parameters and reducing computational cost. Reducing the same number of parameters in different positions in the network may reduce different computational cost. To address the above problems, we develop a novel algorithm named as COP (correlation-based pruning), which can detect the redundant filters efficiently. We enable the cross-layer filter comparison through global normalization. We add parameter-quantity and computational-cost regularization terms to the importance, which enables the users to customize the compression according to their preference (smaller or faster). Extensive experiments have shown COP outperforms the others significantly. The code is released at https://github.com/ZJULearning/COP.

scholarly journals LightOCT: Exploring the depth for Retinal disease detection

2021 ◽  
Author(s):  
Amandeep Kaur ◽  
Vinayak Singh ◽  
Gargi Chakraverty
Keyword(s):  
Neural Network ◽  
Network Architecture ◽  
State Of The Art ◽  
Computational Cost ◽  
Retinal Disease ◽  
Retinal Damage ◽  
Training Time ◽  
Deep Model ◽  
Precise Diagnosis ◽  
Future Work

With the advancement in technology and computation capabilities, identifying retinal damage through state-of-the-art CNNs architectures has led to the speedy and precise diagnosis, thus inhibiting further disease development. In this study, we focus on the classification of retinal damage caused by detecting choroidal neovascularization (CNV), diabetic macular edema (DME), DRUSEN, and NORMAL in optical coherence tomography (OCT) images. The emphasis of our experiment is to investigate the component of depth in the neural network architecture. We introduce a shallow convolution neural network - LightOCT, outperforming the other deep model configurations, with the lowest value of LVCEL and highest accuracy (+98\% in each class). Next, we experimented to find the best fit optimizer for LightOCT. The results proved that the combination of LightOCT and Adam gave the most optimal results. Finally, we compare our approach with transfer learning models, and LightOCT outperforms the state-of-the-art models in terms of computational cost, least training time and gives comparable results in the criteria of accuracy. We would direct our future work to improve the accuracy metrics with shallow models such that the trade-off between training time and accuracy is reduced.

scholarly journals DEEP LEARNING BASED SKIN LESIONS DIAGNOSIS

2019 ◽  
Vol XLII-2/W12 ◽  
pp. 81-85 ◽  
Author(s):  
D. A. Gavrilov ◽  
N. N. Shchelkunov ◽  
A. V. Melerzanov
Keyword(s):  
Neural Network ◽  
Classification Accuracy ◽  
Intelligent Systems ◽  
State Of The Art ◽  
Skin Lesions ◽  
Deep Convolutional Neural Networks ◽  
Melanoma Diagnosis ◽  
Malignant Skin ◽  
Diagnosis Accuracy ◽  
Visual Diagnosis

<p><strong>Abstract.</strong> Melanoma is one of the most virulent lesions of human’s skin. The visual diagnosis accuracy of melanoma directly depends on the doctor’s qualification and specialization. State-of-the-art solutions in the field of image processing and machine learning allows to create intelligent systems based on artificial convolutional neural network exceeding human’s rates in the field of object classification, including the case of malignant skin lesions. This paper presents an algorithm for the early melanoma diagnosis based on artificial deep convolutional neural networks. The algorithm proposed allows to reach the classification accuracy of melanoma at least 91%.</p>

scholarly journals A Normalized Convolutional Neural Network for Guided Sparse Depth Upsampling

2018 ◽  
Author(s):  
Jiashen Hua ◽  
Xiaojin Gong
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Color Image ◽  
State Of The Art ◽  
Depth Map ◽  
Sparse Data ◽  
Optimal Choice ◽  
Deep Convolutional Neural Networks ◽  
Normalized Convolution ◽  
Indoor And Outdoor

Guided sparse depth upsampling aims to upsample an irregularly sampled sparse depth map when an aligned high-resolution color image is given as guidance. When deep convolutional neural networks (CNNs) become the optimal choice to many applications nowadays, how to deal with irregular and sparse data still remains a non-trivial problem. Inspired by the classical normalized convolution operation, this work proposes a normalized convolutional layer (NCL) implemented in CNNs. Sparse data are therefore explicitly considered in CNNs by the separation of both data and filters into a signal part and a certainty part. Based upon NCLs, we design a normalized convolutional neural network (NCNN) to perform guided sparse depth upsampling. Experiments on both indoor and outdoor datasets show that the proposed NCNN models achieve state-of-the-art upsampling performance. Moreover, the models using NCLs gain a great generalization ability to different sparsity levels.

scholarly journals Layerwise Sparse Coding for Pruned Deep Neural Networks with Extreme Compression Ratio

2020 ◽  
Vol 34 (04) ◽  
pp. 4900-4907
Author(s):  
Xiao Liu ◽  
Wenbin Li ◽  
Jing Huo ◽  
Lili Yao ◽  
Yang Gao
Keyword(s):  
Compression Ratio ◽  
Sparse Coding ◽  
Sparse Matrix ◽  
Matrix Multiplication ◽  
Compression Technique ◽  
Meta Data ◽  
Relative Index ◽  
Model Compression ◽  
Deep Model ◽  
Network Compression

Deep neural network compression is important and increasingly developed especially in resource-constrained environments, such as autonomous drones and wearable devices. Basically, we can easily and largely reduce the number of weights of a trained deep model by adopting a widely used model compression technique, e.g., pruning. In this way, two kinds of data are usually preserved for this compressed model, i.e., non-zero weights and meta-data, where meta-data is employed to help encode and decode these non-zero weights. Although we can obtain an ideally small number of non-zero weights through pruning, existing sparse matrix coding methods still need a much larger amount of meta-data (may several times larger than non-zero weights), which will be a severe bottleneck of the deploying of very deep models. To tackle this issue, we propose a layerwise sparse coding (LSC) method to maximize the compression ratio by extremely reducing the amount of meta-data. We first divide a sparse matrix into multiple small blocks and remove zero blocks, and then propose a novel signed relative index (SRI) algorithm to encode the remaining non-zero blocks (with much less meta-data). In addition, the proposed LSC performs parallel matrix multiplication without full decoding, while traditional methods cannot. Through extensive experiments, we demonstrate that LSC achieves substantial gains in pruned DNN compression (e.g., 51.03x compression ratio on ADMM-Lenet) and inference computation (i.e., time reduction and extremely less memory bandwidth), over state-of-the-art baselines.

scholarly journals Research on Network Layer Recursive Reduction Model Compression for Image Recognition

2021 ◽  
Vol 2021 ◽  
pp. 1-11
Author(s):  
Hongfei Ling ◽  
Weiwei Zhang ◽  
Yingjie Tao ◽  
Mi Zhou
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Model ◽  
Computational Cost ◽  
Network Size ◽  
Compression Technique ◽  
Network Layer ◽  
Network Layers ◽  
Model Compression ◽  
The Neural Network

ResNet has been widely used in the field of machine learning since it was proposed. This network model is successful in extracting features from input data by superimposing multiple layers of neural networks and thus achieves high accuracy in many applications. However, the superposition of multilayer neural networks increases their computational cost. For this reason, we propose a network model compression technique that removes multiple neural network layers from ResNet without decreasing the accuracy rate. The key idea is to provide a priority term to identify the importance of each neural network layer, and then select the unimportant layers to be removed during the training process based on the priority of the neural network layers. In addition, this paper also retrains the network model to avoid the accuracy degradation caused by the deletion of network layers. Experiments demonstrate that the network size can be reduced by 24.00%–42.86% of the number of layers without reducing the classification accuracy when classification is performed on CIFAR-10/100 and ImageNet.

scholarly journals Logo Recognition with the Use of Deep Convolutional Neural Networks

2020 ◽  
Vol 10 (5) ◽  
pp. 6191-6194
Author(s):  
A. Alsheikhy ◽  
Y. Said ◽  
M. Barr
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Object Recognition ◽  
Neural Network Model ◽  
State Of The Art ◽  
Original Data ◽  
Deep Convolutional Neural Networks ◽  
Convolutional Networks ◽  
Computational Overhead ◽  
Logo Recognition

Automatic logo recognition is gaining importance due to the increasing number of its applications. Unlike other object recognition tasks, logo recognition is more challenging because of the limited amount of the available original data. In this paper, the transfer leaning technique was applied to a Deep Convolutional Neural Network model to guarantee logo recognition using a small computational overhead. The proposed method was based on the Densely Connected Convolutional Networks (DenseNet). The experimental results show that for the FlickrLogos-32 logo recognition dataset, our proposed method performs comparably with state-of-the-art methods while using fewer parameters.

scholarly journals Garments Texture Design Class Identification Using Deep Convolutional Neural Network

2016 ◽  
Author(s):  
S.M. Sofiqul Islam ◽  
Emon Kumar Dey ◽  
Md. Nurul Ahad Tawhid ◽  
B. M. Mainul Hossain
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Object Recognition ◽  
Online Shopping ◽  
State Of The Art ◽  
Deep Convolutional Neural Networks ◽  
Deep Cnn ◽  
Class Identification ◽  
Multiple Levels ◽  
Texture Design

Automatic garments design class identification for recommending the fashion trends is important nowadays because of the rapid growth of online shopping. By learning the properties of images efficiently, a machine can give better accuracy of classification. Several methods, based on Hand-Engineered feature coding exist for identifying garments design classes. But, most of the time, those methods do not help to achieve better results. Recently, Deep Convolutional Neural Networks (CNNs) have shown better performances for different object recognition. Deep CNN uses multiple levels of representation and abstraction that helps a machine to understand the types of data (images, sound, and text) more accurately. In this paper, we have applied deep CNN for identifying garments design classes. To evaluate the performances, we used two well-known CNN models AlexNet and VGGNet on two different datasets. We also propose a new CNN model based on AlexNet and found better results than existing state-of-the-art by a significant margin.

scholarly journals IoTNet: An Efficient and Accurate Convolutional Neural Network for IoT Devices

Sensors ◽  
2019 ◽  
Vol 19 (24) ◽  
pp. 5541 ◽  
Author(s):  
Tom Lawrence ◽  
Li Zhang
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
State Of The Art ◽  
Computational Cost ◽  
Resource Constrained ◽  
Traffic Sign ◽  
Constrained Environments ◽  
Scale Down ◽  
Art Performance ◽  
Iot Devices

Two main approaches exist when deploying a Convolutional Neural Network (CNN) on resource-constrained IoT devices: either scale a large model down or use a small model designed specifically for resource-constrained environments. Small architectures typically trade accuracy for computational cost by performing convolutions as depth-wise convolutions rather than standard convolutions like in large networks. Large models focus primarily on state-of-the-art performance and often struggle to scale down sufficiently. We propose a new model, namely IoTNet, designed for resource-constrained environments which achieves state-of-the-art performance within the domain of small efficient models. IoTNet trades accuracy with computational cost differently from existing methods by factorizing standard 3 × 3 convolutions into pairs of 1 × 3 and 3 × 1 standard convolutions, rather than performing depth-wise convolutions. We benchmark IoTNet against state-of-the-art efficiency-focused models and scaled-down large architectures on data sets which best match the complexity of problems faced in resource-constrained environments. We compare model accuracy and the number of floating-point operations (FLOPs) performed as a measure of efficiency. We report state-of-the-art accuracy improvement over MobileNetV2 on CIFAR-10 of 13.43% with 39% fewer FLOPs, over ShuffleNet on Street View House Numbers (SVHN) of 6.49% with 31.8% fewer FLOPs and over MobileNet on German Traffic Sign Recognition Benchmark (GTSRB) of 5% with 0.38% fewer FLOPs.

scholarly journals A Multiscale-Based Adjustable Convolutional Neural Network for Multiple Organ Segmentation

2020 ◽  
Vol 2020 ◽  
pp. 1-13
Author(s):  
Zhiqiang Tian ◽  
Jingyi Song ◽  
Chenyang Zhang ◽  
Xiaohui Tian ◽  
Zhong Shi ◽  
...  
Keyword(s):  
Neural Network ◽  
State Of The Art ◽  
Organs At Risk ◽  
Multiple Organ ◽  
Medical Image Segmentation ◽  
Model Complexity ◽  
Dice Similarity Coefficient ◽  
Deep Convolutional Neural Networks ◽  
Multiple Organs ◽  
Art Methods

Accurate segmentation ofs organs-at-risk (OARs) in computed tomography (CT) is the key to planning treatment in radiation therapy (RT). Manually delineating OARs over hundreds of images of a typical CT scan can be time-consuming and error-prone. Deep convolutional neural networks with specific structures like U-Net have been proven effective for medical image segmentation. In this work, we propose an end-to-end deep neural network for multiorgan segmentation with higher accuracy and lower complexity. Compared with several state-of-the-art methods, the proposed accuracy-complexity adjustment module (ACAM) can increase segmentation accuracy and reduce the model complexity and memory usage simultaneously. An attention-based multiscale aggregation module (MAM) is also proposed for further improvement. Experiment results on chest CT datasets show that the proposed network achieves competitive Dice similarity coefficient results with fewer float-point operations (FLOPs) for multiple organs, which outperforms several state-of-the-art methods.

scholarly journals Dynamic Residual Dense Network for Image Denoising

Sensors ◽  
2019 ◽  
Vol 19 (17) ◽  
pp. 3809 ◽  
Author(s):  
Yuda Song ◽  
Yunfang Zhu ◽  
Xin Du
Keyword(s):  
Real World ◽  
State Of The Art ◽  
Computational Cost ◽  
Dynamic Network ◽  
Input Image ◽  
Dense Network ◽  
Deep Convolutional Neural Networks ◽  
Great Performance ◽  
Image Noise Reduction ◽  
Better Than

Deep convolutional neural networks have achieved great performance on various image restoration tasks. Specifically, the residual dense network (RDN) has achieved great results on image noise reduction by cascading multiple residual dense blocks (RDBs) to make full use of the hierarchical feature. However, the RDN only performs well in denoising on a single noise level, and the computational cost of the RDN increases significantly with the increase in the number of RDBs, and this only slightly improves the effect of denoising. To overcome this, we propose the dynamic residual dense network (DRDN), a dynamic network that can selectively skip some RDBs based on the noise amount of the input image. Moreover, the DRDN allows modifying the denoising strength to manually get the best outputs, which can make the network more effective for real-world denoising. Our proposed DRDN can perform better than the RDN and reduces the computational cost by 40 – 50 % . Furthermore, we surpass the state-of-the-art CBDNet by 1.34 dB on the real-world noise benchmark.

Sign in / Sign up

Export Citation Format

Share Document