Fast Training Methods and Their Experiments for Deep Learning CNN Models

2021 ◽  
Author(s):  
Shanshan Jiang ◽  
Sheng-Guo Wang
Keyword(s):  
Deep Learning ◽  
Training Methods ◽  
Fast Training

Fast Training of Deep Learning Models over Multiple GPUs

2020 ◽  
Author(s):  
Xiaodong Yi ◽  
Ziyue Luo ◽  
Chen Meng ◽  
Mengdi Wang ◽  
Guoping Long ◽  
...  
Keyword(s):  
Deep Learning ◽  
Learning Models ◽  
Multiple Gpus ◽  
Fast Training

scholarly journals Pedestrian detection algorithm in traffic scene based on weakly supervised hierarchical deep model

2016 ◽  
Vol 14 (1) ◽  
pp. 172988141769231 ◽  
Author(s):  
Yingfeng Cai ◽  
Youguo He ◽  
Hai Wang ◽  
Xiaoqiang Sun ◽  
Long Chen ◽  
...  
Keyword(s):  
Deep Learning ◽  
Pedestrian Detection ◽  
Recognition Rate ◽  
Detection Algorithm ◽  
Training Methods ◽  
Two Dimensional ◽  
Data Set ◽  
Deep Model ◽  
Unsupervised Training ◽  
Weakly Supervised

The emergence and development of deep learning theory in machine learning field provide new method for visual-based pedestrian recognition technology. To achieve better performance in this application, an improved weakly supervised hierarchical deep learning pedestrian recognition algorithm with two-dimensional deep belief networks is proposed. The improvements are made by taking into consideration the weaknesses of structure and training methods of existing classifiers. First, traditional one-dimensional deep belief network is expanded to two-dimensional that allows image matrix to be loaded directly to preserve more information of a sample space. Then, a determination regularization term with small weight is added to the traditional unsupervised training objective function. By this modification, original unsupervised training is transformed to weakly supervised training. Subsequently, that gives the extracted features discrimination ability. Multiple sets of comparative experiments show that the performance of the proposed algorithm is better than other deep learning algorithms in recognition rate and outperforms most of the existing state-of-the-art methods in non-occlusion pedestrian data set while performs fair in weakly and heavily occlusion data set.

Recognition of calcifications in thyroid nodules based on attention-gated collaborative supervision network of ultrasound images

2020 ◽  
Vol 28 (6) ◽  
pp. 1123-1139
Author(s):  
Liqun Zhang ◽  
Ke Chen ◽  
Lin Han ◽  
Yan Zhuang ◽  
Zhan Hua ◽  
...  
Keyword(s):  
Deep Learning ◽  
Object Recognition ◽  
Thyroid Nodules ◽  
Ultrasound Images ◽  
Training Methods ◽  
Important Criterion ◽  
Improve Accuracy ◽  
Important Means ◽  
Model Training ◽  
Medical Dataset

BACKGROUND: Calcification is an important criterion for classification between benign and malignant thyroid nodules. Deep learning provides an important means for automatic calcification recognition, but it is tedious to annotate pixel-level labels for calcifications with various morphologies. OBJECTIVE: This study aims to improve accuracy of calcification recognition and prediction of its location, as well as to reduce the number of pixel-level labels in model training. METHODS: We proposed a collaborative supervision network based on attention gating (CS-AGnet), which was composed of two branches: a segmentation network and a classification network. The reorganized two-stage collaborative semi-supervised model was trained under the supervision of all image-level labels and few pixel-level labels. RESULTS: The results show that although our semi-supervised network used only 30% (289 cases) of pixel-level labels for training, the accuracy of calcification recognition reaches 92.1%, which is very close to 92.9% of deep supervision with 100% (966 cases) pixel-level labels. The CS-AGnet enables to focus the model’s attention on calcification objects. Thus, it achieves higher accuracy than other deep learning methods. CONCLUSIONS: Our collaborative semi-supervised model has a preferable performance in calcification recognition, and it reduces the number of manual annotations of pixel-level labels. Moreover, it may be of great reference for the object recognition of medical dataset with few labels.

scholarly journals The Effectiveness of Using a Pretrained Deep Learning Neural Networks for Object Classification in Underwater Video

2020 ◽  
Vol 12 (18) ◽  
pp. 3020
Author(s):  
Piotr Szymak ◽  
Paweł Piskur ◽  
Krzysztof Naus
Keyword(s):  
Deep Learning ◽  
World War Ii ◽  
Graphics Processing Units ◽  
Object Classification ◽  
Underwater Vehicles ◽  
Training Methods ◽  
Underwater Video ◽  
Deep Learning Neural Network ◽  
Graphics Processing

Video image processing and object classification using a Deep Learning Neural Network (DLNN) can significantly increase the autonomy of underwater vehicles. This paper describes the results of a project focused on using DLNN for Object Classification in Underwater Video (OCUV) implemented in a Biomimetic Underwater Vehicle (BUV). The BUV is intended to be used to detect underwater mines, explore shipwrecks or observe the process of corrosion of munitions abandoned on the seabed after World War II. Here, the pretrained DLNNs were used for classification of the following type of objects: fishes, underwater vehicles, divers and obstacles. The results of our research enabled us to estimate the effectiveness of using pretrained DLNNs for classification of different objects under the complex Baltic Sea environment. The Genetic Algorithm (GA) was used to establish tuning parameters of the DLNNs. Three different training methods were compared for AlexNet, then one training method was chosen for fifteen networks and the tests were provided with the description of the final results. The DLNNs were trained on servers with six medium class Graphics Processing Units (GPUs). Finally, the trained DLNN was implemented in the Nvidia JetsonTX2 platform installed on board of the BUV, and one of the network was verified in a real environment.

scholarly journals Deep Convolutional Extreme Learning Machine and Its Application in Handwritten Digit Classification

2016 ◽  
Vol 2016 ◽  
pp. 1-10 ◽  
Author(s):  
Shan Pang ◽  
Xinyi Yang
Keyword(s):  
Deep Learning ◽  
Extreme Learning Machine ◽  
Neuron Network ◽  
Learning Methods ◽  
Training Time ◽  
Fast Training ◽  
Handwritten Digit ◽  
Learning Machine ◽  
Hidden Layer ◽  
High Level

In recent years, some deep learning methods have been developed and applied to image classification applications, such as convolutional neuron network (CNN) and deep belief network (DBN). However they are suffering from some problems like local minima, slow convergence rate, and intensive human intervention. In this paper, we propose a rapid learning method, namely, deep convolutional extreme learning machine (DC-ELM), which combines the power of CNN and fast training of ELM. It uses multiple alternate convolution layers and pooling layers to effectively abstract high level features from input images. Then the abstracted features are fed to an ELM classifier, which leads to better generalization performance with faster learning speed. DC-ELM also introduces stochastic pooling in the last hidden layer to reduce dimensionality of features greatly, thus saving much training time and computation resources. We systematically evaluated the performance of DC-ELM on two handwritten digit data sets: MNIST and USPS. Experimental results show that our method achieved better testing accuracy with significantly shorter training time in comparison with deep learning methods and other ELM methods.

scholarly journals SNR: S queezing N umerical R ange Defuses Bit Error Vulnerability Surface in Deep Neural Networks

2021 ◽  
Vol 20 (5s) ◽  
pp. 1-25
Author(s):  
Elbruz Ozen ◽  
Alex Orailoglu
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Deep Neural Networks ◽  
Low Cost ◽  
Error Resilience ◽  
Error Rates ◽  
Training Methods ◽  
Performance Requirements ◽  
Proper Design ◽  
And Performance

As deep learning algorithms are widely adopted, an increasing number of them are positioned in embedded application domains with strict reliability constraints. The expenditure of significant resources to satisfy performance requirements in deep neural network accelerators has thinned out the margins for delivering safety in embedded deep learning applications, thus precluding the adoption of conventional fault tolerance methods. The potential of exploiting the inherent resilience characteristics of deep neural networks remains though unexplored, offering a promising low-cost path towards safety in embedded deep learning applications. This work demonstrates the possibility of such exploitation by juxtaposing the reduction of the vulnerability surface through the proper design of the quantization schemes with shaping the parameter distributions at each layer through the guidance offered by appropriate training methods, thus delivering deep neural networks of high resilience merely through algorithmic modifications. Unequaled error resilience characteristics can be thus injected into safety-critical deep learning applications to tolerate bit error rates of up to at absolutely zero hardware, energy, and performance costs while improving the error-free model accuracy even further.

scholarly journals Local Regularizer Improves Generalization

2020 ◽  
Vol 34 (04) ◽  
pp. 6861-6868 ◽  
Author(s):  
Yikai Zhang ◽  
Hui Qu ◽  
Dimitris Metaxas ◽  
Chao Chen
Keyword(s):  
Deep Learning ◽  
Theoretical Analysis ◽  
Experimental Evidence ◽  
Gradient Descent ◽  
Stochastic Gradient ◽  
Stochastic Gradient Descent ◽  
Training Algorithms ◽  
Training Methods ◽  
Theoretical Understanding ◽  
Better Than

Regularization plays an important role in generalization of deep learning. In this paper, we study the generalization power of an unbiased regularizor for training algorithms in deep learning. We focus on training methods called Locally Regularized Stochastic Gradient Descent (LRSGD). An LRSGD leverages a proximal type penalty in gradient descent steps to regularize SGD in training. We show that by carefully choosing relevant parameters, LRSGD generalizes better than SGD. Our thorough theoretical analysis is supported by experimental evidence. It advances our theoretical understanding of deep learning and provides new perspectives on designing training algorithms. The code is available at https://github.com/huiqu18/LRSGD.

scholarly journals PreNNsem: A Heterogeneous Ensemble Learning Framework for Vulnerability Detection in Software

2020 ◽  
Vol 10 (22) ◽  
pp. 7954
Author(s):  
Lu Wang ◽  
Xin Li ◽  
Ruiheng Wang ◽  
Yang Xin ◽  
Mingcheng Gao ◽  
...  
Keyword(s):  
Deep Learning ◽  
Ensemble Learning ◽  
Detection Efficiency ◽  
False Positive Rate ◽  
False Negative ◽  
False Negative Rate ◽  
Training Methods ◽  
Vulnerability Detection ◽  
Sample Data ◽  
Heterogeneous Ensemble

Automated vulnerability detection is one of the critical issues in the realm of software security. Existing solutions to this problem are mostly based on features that are defined by human experts and directly lead to missed potential vulnerability. Deep learning is an effective method for automating the extraction of vulnerability characteristics. Our paper proposes intelligent and automated vulnerability detection while using deep representation learning and heterogeneous ensemble learning. Firstly, we transform sample data from source code by removing segments that are unrelated to the vulnerability in order to reduce code analysis and improve detection efficiency in our experiments. Secondly, we represent the sample data as real vectors by pre-training on the corpus and maintaining its semantic information. Thirdly, the vectors are fed to a deep learning model to obtain the features of vulnerability. Lastly, we train a heterogeneous ensemble classifier. We analyze the effectiveness and resource consumption of different network models, pre-training methods, classifiers, and vulnerabilities separately in order to evaluate the detection method. We also compare our approach with some well-known vulnerability detection commercial tools and academic methods. The experimental results show that our proposed method provides improvements in false positive rate, false negative rate, precision, recall, and F1 score.

scholarly journals Explainable Artificial Intelligence for Human-Machine Interaction in Brain Tumor Localization

2021 ◽  
Vol 11 (11) ◽  
pp. 1213
Author(s):  
Morteza Esmaeili ◽  
Riyas Vettukattil ◽  
Hasan Banitalebi ◽  
Nina R. Krogh ◽  
Jonn Terje Geitung
Keyword(s):  
Artificial Intelligence ◽  
Deep Learning ◽  
Training Methods ◽  
Human Machine Interaction ◽  
Localization Accuracy ◽  
Explainable Ai ◽  
Recent Developments ◽  
High Level ◽  
Machine Interaction ◽  
Selection Of

Primary malignancies in adult brains are globally fatal. Computer vision, especially recent developments in artificial intelligence (AI), have created opportunities to automatically characterize and diagnose tumor lesions in the brain. AI approaches have provided scores of unprecedented accuracy in different image analysis tasks, including differentiating tumor-containing brains from healthy brains. AI models, however, perform as a black box, concealing the rational interpretations that are an essential step towards translating AI imaging tools into clinical routine. An explainable AI approach aims to visualize the high-level features of trained models or integrate into the training process. This study aims to evaluate the performance of selected deep-learning algorithms on localizing tumor lesions and distinguishing the lesion from healthy regions in magnetic resonance imaging contrasts. Despite a significant correlation between classification and lesion localization accuracy (R = 0.46, p = 0.005), the known AI algorithms, examined in this study, classify some tumor brains based on other non-relevant features. The results suggest that explainable AI approaches can develop an intuition for model interpretability and may play an important role in the performance evaluation of deep learning models. Developing explainable AI approaches will be an essential tool to improve human–machine interactions and assist in the selection of optimal training methods.

Sign in / Sign up

Export Citation Format

Share Document