An improved model training method for residual convolutional neural networks in deep learning

Face recognition algorithms based on deep learning methods have become increasingly popular. Most of these are based on highly precise but complex convolutional neural networks (CNNs), which require significant computing resources and storage, and are difficult to deploy on mobile devices or embedded terminals. In this paper, we propose several methods to improve the algorithms for face recognition based on a lightweight CNN, which is further optimized in terms of the network architecture and training pattern on the basis of MobileFaceNet. Regarding the network architecture, we introduce the Squeeze-and-Excitation (SE) block and propose three improved structures via a channel attention mechanism—the depthwise SE module, the depthwise separable SE module, and the linear SE module—which are able to learn the correlation of information between channels and assign them different weights. In addition, a novel training method for the face recognition task combined with an additive angular margin loss function is proposed that performs the compression and knowledge transfer of the deep network for face recognition. Finally, we obtained high-precision and lightweight face recognition models with fewer parameters and calculations that are more suitable for applications. Through extensive experiments and analysis, we demonstrate the effectiveness of the proposed methods.

Download Full-text

A Training Method for Low Rank Convolutional Neural Networks Based on Alternating Tensor Compose-Decompose Method

Applied Sciences ◽

10.3390/app11020643 ◽

2021 ◽

Vol 11 (2) ◽

pp. 643

Author(s):

Sukho Lee ◽

Hyein Kim ◽

Byeongseon Jeong ◽

Jungho Yoon

Keyword(s):

Neural Networks ◽

Computer Vision ◽

Deep Learning ◽

Convolutional Neural Networks ◽

Learning Model ◽

Low Rank ◽

Training Method ◽

Training Stage ◽

Fixed Structure ◽

Deep Learning Model

Over the past decade, deep learning-based computer vision methods have been shown to surpass previous state-of-the-art computer vision techniques in various fields, and have made great progress in various computer vision problems, including object detection, object segmentation, face recognition, etc. Nowadays, major IT companies are adding new deep-learning-based computer technologies to edge devices such as smartphones. However, since the computational cost of deep learning-based models is still high for edge devices, research is being actively carried out to compress deep learning-based models while not sacrificing high performance. Recently, many lightweight architectures have been proposed for deep learning-based models which are based on low-rank approximation. In this paper, we propose an alternating tensor compose-decompose (ATCD) method for the training of low-rank convolutional neural networks. The proposed training method can better train a compressed low-rank deep learning model than the conventional fixed-structure based training method, so that a compressed deep learning model with higher performance can be obtained in the end of the training. As a representative and exemplary model to which the proposed training method can be applied, we propose a rank-1 convolutional neural network (CNN) which has a structure alternatively containing 3-D rank-1 filters and 1-D filters in the training stage and a 1-D structure in the testing stage. After being trained, the 3-D rank-1 filters can be permanently decomposed into 1-D filters to achieve a fast inference in the test time. The reason that the 1-D filters are not being trained directly in 1-D form in the training stage is that the training of the 3-D rank-1 filters is easier due to the better gradient flow, which makes the training possible even in the case when the fixed structured network with fixed consecutive 1-D filters cannot be trained at all. We also show that the same training method can be applied to the well-known MobileNet architecture so that better parameters can be obtained than with the conventional fixed-structure training method. Furthermore, we show that the 1-D filters in a ResNet like structure can also be trained with the proposed method, which shows the fact that the proposed method can be applied to various structures of networks.

Download Full-text

Deep Learning through Convolutional Neural Networks for Classification of Image A Novel Approach Using Hyper Filter

International Journal of Computer Sciences and Engineering ◽

10.26438/ijcse/v7i6.164168 ◽

2019 ◽

Vol 7 (6) ◽

pp. 164-168

Author(s):

Kshitij Tripathi ◽

Rajendra G. Vyas ◽

Anil K. Gupta

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Convolutional Neural Networks ◽

Novel Approach

Download Full-text

CONVOLUTIONAL NEURAL NETWORKS, ANALYTICAL ALGORITHMS, AND PERSONALIZED HEALTH CARE: EMBRACING THE MASSIVE DATA ANALYSIS CAPABILITIES OF DEEP LEARNING ARTIFICIAL INTELLIGENCE SYSTEMS TO COMPLEMENT AND IMPROVE MEDICAL SERVICES

American Journal of Medical Research ◽

10.22381/ajmr5220187 ◽

2018 ◽

Vol 5 (2) ◽

pp. 52 ◽

Cited By ~ 1

Keyword(s):

Artificial Intelligence ◽

Neural Networks ◽

Health Care ◽

Deep Learning ◽

Data Analysis ◽

Convolutional Neural Networks ◽

Massive Data ◽

Personalized Health ◽

Personalized Health Care ◽

Artificial Intelligence Systems

Download Full-text

Deep convolutional neural networks for cardiovascular vulnerable plaque detection

MATEC Web of Conferences ◽

10.1051/matecconf/201927702024 ◽

2019 ◽

Vol 277 ◽

pp. 02024 ◽

Cited By ~ 1

Author(s):

Lincan Li ◽

Tong Jia ◽

Tianqi Meng ◽

Yizhe Liu

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Convolutional Neural Networks ◽

Vulnerable Plaque ◽

Recall Rate ◽

Superior Performance ◽

Learning Approaches ◽

Deep Convolutional Neural Networks ◽

Vulnerable Plaques ◽

Plaque Detection

In this paper, an accurate two-stage deep learning method is proposed to detect vulnerable plaques in ultrasonic images of cardiovascular. Firstly, a Fully Convonutional Neural Network (FCN) named U-Net is used to segment the original Intravascular Optical Coherence Tomography (IVOCT) cardiovascular images. We experiment on different threshold values to find the best threshold for removing noise and background in the original images. Secondly, a modified Faster RCNN is adopted to do precise detection. The modified Faster R-CNN utilize six-scale anchors (122,162,322,642,1282,2562) instead of the conventional one scale or three scale approaches. First, we present three problems in cardiovascular vulnerable plaque diagnosis, then we demonstrate how our method solve these problems. The proposed method in this paper apply deep convolutional neural networks to the whole diagnostic procedure. Test results show the Recall rate, Precision rate, IoU (Intersection-over-Union) rate and Total score are 0.94, 0.885, 0.913 and 0.913 respectively, higher than the 1st team of CCCV2017 Cardiovascular OCT Vulnerable Plaque Detection Challenge. AP of the designed Faster RCNN is 83.4%, higher than conventional approaches which use one-scale or three-scale anchors. These results demonstrate the superior performance of our proposed method and the power of deep learning approaches in diagnose cardiovascular vulnerable plaques.

Download Full-text

Deep Malaria Parasite Detection in Thin Blood Smear Microscopic Images

Applied Sciences ◽

10.3390/app11052284 ◽

2021 ◽

Vol 11 (5) ◽

pp. 2284

Author(s):

Asma Maqsood ◽

Muhammad Shahid Farid ◽

Muhammad Hassan Khan ◽

Marcin Grzegorzek

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Red Blood Cells ◽

Convolutional Neural Networks ◽

Blood Cells ◽

Superior Performance ◽

Learning Models ◽

Infected Female ◽

The World ◽

Augmentation Techniques

Malaria is a disease activated by a type of microscopic parasite transmitted from infected female mosquito bites to humans. Malaria is a fatal disease that is endemic in many regions of the world. Quick diagnosis of this disease will be very valuable for patients, as traditional methods require tedious work for its detection. Recently, some automated methods have been proposed that exploit hand-crafted feature extraction techniques however, their accuracies are not reliable. Deep learning approaches modernize the world with their superior performance. Convolutional Neural Networks (CNN) are vastly scalable for image classification tasks that extract features through hidden layers of the model without any handcrafting. The detection of malaria-infected red blood cells from segmented microscopic blood images using convolutional neural networks can assist in quick diagnosis, and this will be useful for regions with fewer healthcare experts. The contributions of this paper are two-fold. First, we evaluate the performance of different existing deep learning models for efficient malaria detection. Second, we propose a customized CNN model that outperforms all observed deep learning models. It exploits the bilateral filtering and image augmentation techniques for highlighting features of red blood cells before training the model. Due to image augmentation techniques, the customized CNN model is generalized and avoids over-fitting. All experimental evaluations are performed on the benchmark NIH Malaria Dataset, and the results reveal that the proposed algorithm is 96.82% accurate in detecting malaria from the microscopic blood smears.

Download Full-text

Efficient downscaling of satellite oceanographic data with convolutional neural networks

SIGSPATIAL Special ◽

10.1145/3447994.3448011 ◽

2021 ◽

Vol 12 (3) ◽

pp. 46-47

Author(s):

Nikita Saxena

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Convolutional Neural Networks ◽

Super Resolution ◽

Sea Surface ◽

Clear Sky ◽

Oceanographic Data ◽

Computationally Expensive ◽

Sky Conditions ◽

Data Inference

Space-borne satellite radiometers measure Sea Surface Temperature (SST), which is pivotal to studies of air-sea interactions and ocean features. Under clear sky conditions, high resolution measurements are obtainable. But under cloudy conditions, data analysis is constrained to the available low resolution measurements. We assess the efficiency of Deep Learning (DL) architectures, particularly Convolutional Neural Networks (CNN) to downscale oceanographic data from low spatial resolution (SR) to high SR. With a focus on SST Fields of Bay of Bengal, this study proves that Very Deep Super Resolution CNN can successfully reconstruct SST observations from 15 km SR to 5km SR, and 5km SR to 1km SR. This outcome calls attention to the significance of DL models explicitly trained for the reconstruction of high SR SST fields by using low SR data. Inference on DL models can act as a substitute to the existing computationally expensive downscaling technique: Dynamical Downsampling. The complete code is available on this Github Repository.

Download Full-text