Learning a deep convolutional neural network via tensor decomposition

2021 ◽  
Author(s):  
Samet Oymak ◽  
Mahdi Soltanolkotabi
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Sample Size ◽  
Gaussian Distribution ◽  
Tensor Decomposition ◽  
Deep Convolutional Neural Network ◽  
Training Data ◽  
Low Rank ◽  
Rank Tensor

Abstract In this paper, we study the problem of learning the weights of a deep convolutional neural network. We consider a network where convolutions are carried out over non-overlapping patches. We develop an algorithm for simultaneously learning all the kernels from the training data. Our approach dubbed deep tensor decomposition (DeepTD) is based on a low-rank tensor decomposition. We theoretically investigate DeepTD under a realizable model for the training data where the inputs are chosen i.i.d. from a Gaussian distribution and the labels are generated according to planted convolutional kernels. We show that DeepTD is sample efficient and provably works as soon as the sample size exceeds the total number of convolutional weights in the network.

scholarly journals Stable Low-Rank Tensor Decomposition for Compression of Convolutional Neural Network

2020 ◽  
pp. 522-539
Author(s):  
Anh-Huy Phan ◽  
Konstantin Sobolev ◽  
Konstantin Sozykin ◽  
Dmitry Ermilov ◽  
Julia Gusak ◽  
...  
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Tensor Decomposition ◽  
Low Rank ◽  
Rank Tensor

scholarly journals Object Detection in Ground-Penetrating Radar Images Using a Deep Convolutional Neural Network and Image Set Preparation by Migration

2018 ◽  
Vol 2018 ◽  
pp. 1-8 ◽  
Author(s):  
Kazuya Ishitsuka ◽  
Shinichiro Iso ◽  
Kyosuke Onishi ◽  
Toshifumi Matsuoka
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Ground Penetrating Radar ◽  
Deep Convolutional Neural Network ◽  
Training Data ◽  
Neural Network Approach ◽  
Radar Images ◽  
Image Set ◽  
Ground Penetrating ◽  
Better Than

Ground-penetrating radar allows the acquisition of many images for investigation of the pavement interior and shallow geological structures. Accordingly, an efficient methodology of detecting objects, such as pipes, reinforcing steel bars, and internal voids, in ground-penetrating radar images is an emerging technology. In this paper, we propose using a deep convolutional neural network to detect characteristic hyperbolic signatures from embedded objects. As a first step, we developed a migration-based method to collect many training data and created 53510 categorized images. We then examined the accuracy of the deep convolutional neural network in detecting the signatures. The accuracy of the classification was 0.945 (94.5%)–0.979 (97.9%) when using several thousands of training images and was much better than the accuracy of the conventional neural network approach. Our results demonstrate the effectiveness of the deep convolutional neural network in detecting characteristic events in ground-penetrating radar images.

scholarly journals Multidimensional Face Representation in a Deep Convolutional Neural Network Reveals the Mechanism Underlying AI Racism

2021 ◽  
Vol 15 ◽  
Author(s):  
Jinhua Tian ◽  
Hailun Xie ◽  
Siyuan Hu ◽  
Jia Liu
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Underlying Mechanism ◽  
Deep Convolutional Neural Network ◽  
The Other ◽  
Training Data ◽  
Identification Accuracy ◽  
Social Bias ◽  
Novel Approach ◽  
Ai Ethics

The increasingly popular application of AI runs the risk of amplifying social bias, such as classifying non-white faces as animals. Recent research has largely attributed this bias to the training data implemented. However, the underlying mechanism is poorly understood; therefore, strategies to rectify the bias are unresolved. Here, we examined a typical deep convolutional neural network (DCNN), VGG-Face, which was trained with a face dataset consisting of more white faces than black and Asian faces. The transfer learning result showed significantly better performance in identifying white faces, similar to the well-known social bias in humans, the other-race effect (ORE). To test whether the effect resulted from the imbalance of face images, we retrained the VGG-Face with a dataset containing more Asian faces, and found a reverse ORE that the newly-trained VGG-Face preferred Asian faces over white faces in identification accuracy. Additionally, when the number of Asian faces and white faces were matched in the dataset, the DCNN did not show any bias. To further examine how imbalanced image input led to the ORE, we performed a representational similarity analysis on VGG-Face's activation. We found that when the dataset contained more white faces, the representation of white faces was more distinct, indexed by smaller in-group similarity and larger representational Euclidean distance. That is, white faces were scattered more sparsely in the representational face space of the VGG-Face than the other faces. Importantly, the distinctiveness of faces was positively correlated with identification accuracy, which explained the ORE observed in the VGG-Face. In summary, our study revealed the mechanism underlying the ORE in DCNNs, which provides a novel approach to studying AI ethics. In addition, the face multidimensional representation theory discovered in humans was also applicable to DCNNs, advocating for future studies to apply more cognitive theories to understand DCNNs' behavior.

CT Image Segmentation of Liver Tumor with Deep Convolutional Neural Network

2021 ◽  
Vol 11 (2) ◽  
pp. 337-344
Author(s):  
Yao Zeng ◽  
Huanhuan Dai
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Automatic Segmentation ◽  
Abdominal Cavity ◽  
Mutual Exclusion ◽  
Image Data ◽  
Deep Convolutional Neural Network ◽  
Training Data ◽  
Segmentation Method ◽  
Segmentation Boundary

The liver is the largest substantial organ in the abdominal cavity of the human body. Its structure is complex, the incidence of vascular abundance is high, and it has been seriously ribbed, human health and life. In this study, an automatic segmentation method based on deep convolutional neural network is proposed. Image data blocks of different sizes are extracted as training data and different network structures are designed, and features are automatically learned to obtain a segmentation structure of the tumor. Secondly, in order to further refine the segmentation boundary, we establish a multi-region segmentation model with region mutual exclusion constraints. The model combines the image grayscale, gradient and prior probability information, and overcomes the problem that the boundary point attribution area caused by boundary blur and regional adhesion is difficult to determine. Finally, the model is solved quickly using the time-invisible multi-phase level set. Compared with the traditional multi-organ segmentation method, this method does not require registration or model initialization. The experimental results show that the model can segment the liver, kidney and spleen quickly and effectively, and the segmentation accuracy reaches the advanced level of current methods.

scholarly journals The Effect of Different Flaw Data to Machine Learning Powered Ultrasonic Inspection

2021 ◽  
Vol 40 (1) ◽  
Author(s):  
Tuomas Koskinen ◽  
Iikka Virkkunen ◽  
Oskar Siljama ◽  
Oskari Jessen-Juhler
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Convolutional Neural Network ◽  
Image Recognition ◽  
Phased Array ◽  
Ultrasonic Inspection ◽  
Deep Convolutional Neural Network ◽  
Flaw Size ◽  
Training Data ◽  
Link Type

AbstractPrevious research (Li et al., Understanding the disharmony between dropout and batch normalization by variance shift. CoRR abs/1801.05134 (2018). http://arxiv.org/abs/1801.05134arXiv:1801.05134) has shown the plausibility of using a modern deep convolutional neural network to detect flaws from phased-array ultrasonic data. This brings the repeatability and effectiveness of automated systems to complex ultrasonic signal evaluation, previously done exclusively by human inspectors. The major breakthrough was to use virtual flaws to generate ample flaw data for the teaching of the algorithm. This enabled the use of raw ultrasonic scan data for detection and to leverage some of the approaches used in machine learning for image recognition. Unlike traditional image recognition, training data for ultrasonic inspection is scarce. While virtual flaws allow us to broaden the data considerably, original flaws with proper flaw-size distribution are still required. This is of course the same for training human inspectors. The training of human inspectors is usually done with easily manufacturable flaws such as side-drilled holes and EDM notches. While the difference between these easily manufactured artificial flaws and real flaws is obvious, human inspectors still manage to train with them and perform well in real inspection scenarios. In the present work, we use a modern, deep convolutional neural network to detect flaws from phased-array ultrasonic data and compare the results achieved from different training data obtained from various artificial flaws. The model demonstrated good generalization capability toward flaw sizes larger than the original training data, and the effect of the minimum flaw size in the data set affects the $$a_{90/95}$$ a 90 / 95 value. This work also demonstrates how different artificial flaws, solidification cracks, EDM notch and simple simulated flaws generalize differently.

scholarly journals Impact of Training Sample Size on the Effects of Regularization in a Convolutional Neural Network-based Dental X-ray Artifact Prediction Model

2020 ◽  
Vol 14 (1) ◽  
pp. 5
Author(s):  
Adam Adli ◽  
Pascal Tyrrell
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Convolutional Neural Network ◽  
Sample Size ◽  
Training Sample ◽  
Training Data ◽  
Classification Model ◽  
Sample Sizes ◽  
X Ray ◽  
Training Sample Size

Introduction: Advances in computers have allowed for the practical application of increasingly advanced machine learning models to aid healthcare providers with diagnosis and inspection of medical images. Often, a lack of training data and computation time can be a limiting factor in the development of an accurate machine learning model in the domain of medical imaging. As a possible solution, this study investigated whether L2 regularization moderate s the overfitting that occurs as a result of small training sample sizes.Methods: This study employed transfer learning experiments on a dental x-ray binary classification model to explore L2 regularization with respect to training sample size in five common convolutional neural network architectures. Model testing performance was investigated and technical implementation details including computation times and hardware considerations as well as performance factors and practical feasibility were described.Results: The experimental results showed a trend that smaller training sample sizes benefitted more from regularization than larger training sample sizes. Further, the results showed that applying L2 regularization did not apply significant computational overhead and that the extra rounds of training L2 regularization were feasible when training sample sizes are relatively small.Conclusion: Overall, this study found that there is a window of opportunity in which the benefits of employing regularization can be most cost-effective relative to training sample size. It is recommended that training sample size should be carefully considered when forming expectations of achievable generalizability improvements that result from investing computational resources into model regularization.

Sign in / Sign up

Export Citation Format

Share Document