scholarly journals AN APPROACH TO ACCELERATE THE TRAINING OF CONVOLUTIONAL NEURAL NETWORKS BY TUNING THE HYPERPARAMETERS OF LEARNING

2020 ◽  
Vol 2 (2) ◽  
pp. 32-37
Author(s):  
P. RADIUK ◽  
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Video Processing ◽  
Network Architecture ◽  
Random Search ◽  
Machine Learning Algorithms ◽  
Language Recognition ◽  
Data Set ◽  
Training Time

Over the last decade, a set of machine learning algorithms called deep learning has led to significant improvements in computer vision, natural language recognition and processing. This has led to the widespread use of a variety of commercial, learning-based products in various fields of human activity. Despite this success, the use of deep neural networks remains a black box. Today, the process of setting hyperparameters and designing a network architecture requires experience and a lot of trial and error and is based more on chance than on a scientific approach. At the same time, the task of simplifying deep learning is extremely urgent. To date, no simple ways have been invented to establish the optimal values of learning hyperparameters, namely learning speed, sample size, data set, learning pulse, and weight loss. Grid search and random search of hyperparameter space are extremely resource intensive. The choice of hyperparameters is critical for the training time and the final result. In addition, experts often choose one of the standard architectures (for example, ResNets and ready-made sets of hyperparameters. However, such kits are usually suboptimal for specific practical tasks. The presented work offers an approach to finding the optimal set of hyperparameters of learning ZNM. An integrated approach to all hyperparameters is valuable because there is an interdependence between them. The aim of the work is to develop an approach for setting a set of hyperparameters, which will reduce the time spent during the design of ZNM and ensure the efficiency of its work. In recent decades, the introduction of deep learning methods, in particular convolutional neural networks (CNNs), has led to impressive success in image and video processing. However, the training of CNN has been commonly mostly based on the employment of quasi-optimal hyperparameters. Such an approach usually requires huge computational and time costs to train the network and does not guarantee a satisfactory result. However, hyperparameters play a crucial role in the effectiveness of CNN, as diverse hyperparameters lead to models with significantly different characteristics. Poorly selected hyperparameters generally lead to low model performance. The issue of choosing optimal hyperparameters for CNN has not been resolved yet. The presented work proposes several practical approaches to setting hyperparameters, which allows reducing training time and increasing the accuracy of the model. The article considers the function of training validation loss during underfitting and overfitting. There are guidelines in the end to reach the optimization point. The paper also considers the regulation of learning rate and momentum to accelerate network training. All experiments are based on the widespread CIFAR-10 and CIFAR-100 datasets.

scholarly journals A Research on Deep Learning Advance for Landslide Classification using Convolutional Neural Networks

2019 ◽  
Vol 8 (6S4) ◽  
pp. 903-906
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Feature Extraction ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Support Vector ◽  
Data Set ◽  
Proposed Model

Landslides can easily be tragic to human life and property. Increase in the rate of human settlement in the mountains has resulted in safety concerns. Landslides have caused economic loss between 1-2% of the GDP in many developing countries. In this study, we discuss a deep learning approach to detect landslides. Convolutional Neural Networks are used for feature extraction for our proposed model. As there was no source of an exact and precise data set for feature extraction, therefore, a new data set was built for testing the model. We have tested and compared this work with our proposed model and with other machine-learning algorithms such as Logistic Regression, Random Forest, AdaBoost, K-Nearest Neighbors and Support Vector Machine. Our proposed deep learning model produces a classification accuracy of 96.90% outperforming the classical machine-learning algorithms.

scholarly journals Newspaper Article Classification using Machine Learning Techniques

2020 ◽  
Vol 9 (5) ◽  
pp. 872-877
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Machine Learning Techniques ◽  
Learning Approach ◽  
Training Time ◽  
Time Prediction ◽  
Learning Techniques ◽  
Prediction Time ◽  
Newspaper Articles

Newspaper articles offer us insights on several news. They can be one of many categories like sports, politics, Science and Technology etc. Text classification is a need of the day as large uncategorized data is the problem everywhere. Through this study, We intend to compare several algorithms along with data preprocessing approaches to classify the newspaper articles into their respective categories. Convolutional Neural Networks(CNN) is a deep learning approach which is currently a strong competitor to other classification algorithms like SVM, Naive Bayes and KNN. We hence intend to implement Convolutional Neural Networks - a deep learning approach to classify our newspaper articles, develop an understanding of all the algorithms implemented and compare their results. We also attempt to compare the training time, prediction time and accuracies of all the algorithms.

scholarly journals A convolutional neural network for predicting transcriptional regulators of genes in Arabidopsis transcriptome data reveals classification based on positive regulatory interactions

2019 ◽  
Author(s):  
Dan MacLean
Keyword(s):  
Neural Network ◽  
Gene Expression ◽  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Convolutional Neural Networks ◽  
Expression Profiles ◽  
Biological Data ◽  
Expression Data ◽  
Data Set

AbstractGene Regulatory networks that control gene expression are widely studied yet the interactions that make them up are difficult to predict from high throughput data. Deep Learning methods such as convolutional neural networks can perform surprisingly good classifications on a variety of data types and the matrix-like gene expression profiles would seem to be ideal input data for deep learning approaches. In this short study I compiled training sets of expression data using the Arabidopsis AtGenExpress global stress expression data set and known transcription factor-target interactions from the Arabidopsis PLACE database. I built and optimised convolutional neural networks with a best model providing 95 % accuracy of classification on a held-out validation set. Investigation of the activations within this model revealed that classification was based on positive correlation of expression profiles in short sections. This result shows that a convolutional neural network can be used to make classifications and reveal the basis of those calssifications for gene expression data sets, indicating that a convolutional neural network is a useful and interpretable tool for exploratory classification of biological data. The final model is available for download and as a web application.

scholarly journals Telugu and Hindi Script Recognition using Deep learning Techniques

2019 ◽  
Vol 8 (11) ◽  
pp. 1758-1764 ◽  
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Character Recognition ◽  
Research Work ◽  
Indian Languages ◽  
Digital Form ◽  
Training Time ◽  
Learning Techniques ◽  
Handwritten Document

The need for offline handwritten character recognition is intense, yet difficult as the writing varies from person to person and also depends on various other factors connected to the attitude and mood of the person. However, we are able to achieve it by converting the handwritten document into digital form. It has been advanced with introducing convolutional neural networks and is further productive with pre-trained models which have the capacity of decreasing the training time and increasing accuracy of character recognition. Research in recognition of handwritten characters for Indian languages is less when compared to other languages like English, Latin, Chinese etc., mainly because it is a multilingual country. Recognition of Telugu and Hindi characters are more difficult as the script of these languages is mostly cursive and are with more diacritics. So the research work in this line is to have inclination towards accuracy in their recognition. Some research has already been started and is successful up to eighty percent in offline hand written character recognition of Telugu and Hindi. The proposed work focuses on increasing accuracy in less time in recognition of these selected languages and is able to reach the expectant values.

scholarly journals Numerical Investigations on Wave Remote Sensing from Synthetic X-Band Radar Sea Clutter Images by Using Deep Convolutional Neural Networks

2020 ◽  
Vol 12 (7) ◽  
pp. 1117 ◽  
Author(s):  
Wenyang Duan ◽  
Ke Yang ◽  
Limin Huang ◽  
Xuewen Ma
Keyword(s):  
Remote Sensing ◽  
Neural Networks ◽  
Spectral Analysis ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Wave Parameter ◽  
Sea Clutter ◽  
Data Set ◽  
Wave Parameters ◽  
X Band

X-band marine radar is an effective tool for sea wave remote sensing. Conventional physical-based methods for acquiring wave parameters from radar sea clutter images use three-dimensional Fourier transform and spectral analysis. They are limited by some assumptions, empirical formulas and the calibration process while obtaining the modulation transfer function (MTF) and signal-to-noise ratio (SNR). Therefore, further improvement of wave inversion accuracy by using the physical-based method presents a challenge. Inspired by the capability of convolutional neural networks (CNN) in image characteristic processing, a deep-learning inversion method based on deep CNN is proposed. No intermediate step or parameter is needed in the CNN-based method, therefore fewer errors are introduced. Wave parameter inversion models were constructed based on CNN to inverse the wave’s spectral peak period and significant wave height. In the present paper, the numerically simulated X-band radar image data were used for a numerical investigation of wave parameters. Results of the conventional spectral analysis and CNN-based methods were compared and the CNN-based method had a higher accuracy on the same data set. The influence of training strategy on CNN-based inversion models was studied to analyze the dependence of a deep-learning inversion model on training data. Additionally, the effects of target parameters on the inversion accuracy of CNN-based models was also studied.

scholarly journals Reionization Models Classifier using 21cm Map Deep Learning

2017 ◽  
Vol 12 (S333) ◽  
pp. 47-51
Author(s):  
Sultan Hassan ◽  
Adrian Liu ◽  
Saul Kohn ◽  
James E. Aguirre ◽  
Paul La Plante ◽  
...  
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Power Spectrum ◽  
Convolutional Neural Networks ◽  
Network Architecture ◽  
Next Generation ◽  
Simple Network

AbstractNext-generation 21cm observations will enable imaging of reionization on very large scales. These images will contain more astrophysical and cosmological information than the power spectrum, and hence providing an alternative way to constrain the contribution of different reionizing sources populations to cosmic reionization. Using Convolutional Neural Networks, we present a simple network architecture that is sufficient to discriminate between Galaxy-dominated versus AGN-dominated models, even in the presence of simulated noise from different experiments such as the HERA and SKA.

scholarly journals Multiscale Convolutional Neural Networks for Hand Detection

2017 ◽  
Vol 2017 ◽  
pp. 1-13 ◽  
Author(s):  
Shiyang Yan ◽  
Yizhang Xia ◽  
Jeremy S. Smith ◽  
Wenjin Lu ◽  
Bailing Zhang
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
State Of The Art ◽  
Hand Tracking ◽  
Language Recognition ◽  
Hand Detection ◽  
Still Images ◽  
Detection Scheme ◽  
Gesture Analysis

Unconstrained hand detection in still images plays an important role in many hand-related vision problems, for example, hand tracking, gesture analysis, human action recognition and human-machine interaction, and sign language recognition. Although hand detection has been extensively studied for decades, it is still a challenging task with many problems to be tackled. The contributing factors for this complexity include heavy occlusion, low resolution, varying illumination conditions, different hand gestures, and the complex interactions between hands and objects or other hands. In this paper, we propose a multiscale deep learning model for unconstrained hand detection in still images. Deep learning models, and deep convolutional neural networks (CNNs) in particular, have achieved state-of-the-art performances in many vision benchmarks. Developed from the region-based CNN (R-CNN) model, we propose a hand detection scheme based on candidate regions generated by a generic region proposal algorithm, followed by multiscale information fusion from the popular VGG16 model. Two benchmark datasets were applied to validate the proposed method, namely, the Oxford Hand Detection Dataset and the VIVA Hand Detection Challenge. We achieved state-of-the-art results on the Oxford Hand Detection Dataset and had satisfactory performance in the VIVA Hand Detection Challenge.

scholarly journals Reinforcement Learning for Hyperparameter Tuning in Deep Learning-based Side-channel Analysis

2021 ◽  
pp. 677-707
Author(s):  
Jorai Rijsdijk ◽  
Lichao Wu ◽  
Guilherme Perin ◽  
Stjepan Picek
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Reinforcement Learning ◽  
Convolutional Neural Networks ◽  
Random Search ◽  
High Price ◽  
Side Channel ◽  
Q Learning ◽  
Reward Functions

Deep learning represents a powerful set of techniques for profiling sidechannel analysis. The results in the last few years show that neural network architectures like multilayer perceptron and convolutional neural networks give strong attack performance where it is possible to break targets protected with various countermeasures. Considering that deep learning techniques commonly have a plethora of hyperparameters to tune, it is clear that such top attack results can come with a high price in preparing the attack. This is especially problematic as the side-channel community commonly uses random search or grid search techniques to look for the best hyperparameters.In this paper, we propose to use reinforcement learning to tune the convolutional neural network hyperparameters. In our framework, we investigate the Q-Learning paradigm and develop two reward functions that use side-channel metrics. We mount an investigation on three commonly used datasets and two leakage models where the results show that reinforcement learning can find convolutional neural networks exhibiting top performance while having small numbers of trainable parameters. We note that our approach is automated and can be easily adapted to different datasets. Several of our newly developed architectures outperform the current state-of-the-art results. Finally, we make our source code publicly available. https://github.com/AISyLab/Reinforcement-Learning-for-SCA

scholarly journals Pneumonia and Eye Disease Detection using Convolutional Neural Networks

2020 ◽  
Vol 10 (3) ◽  
pp. 5769-5774 ◽  
Author(s):  
P. Chakraborty ◽  
C. Tharini
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Detection System ◽  
Eye Disease ◽  
Medical Professionals ◽  
Disease Detection ◽  
Training Time ◽  
X Ray ◽  
Detection Systems

Automatic disease detection systems based on Convolutional Neural Networks (CNNs) are proposed in this paper for helping the medical professionals in the detection of diseases from scan and X-ray images. CNN based classification helps decision making in a prompt manner with high precision. CNNs are a subset of deep learning which is a branch of Artificial Intelligence. The main advantage of CNNs compared to other deep learning algorithms is that they require minimal pre-processing. In the proposed disease detection system, two medical image datasets consisting of Optical Coherence Tomography (OCT) and chest X-ray images of 1-5 year-old children are considered and used as inputs. The medical images are processed and classified using CNN and various performance measuring parameters such as accuracy, loss, and training time are measured. The system is then implemented in hardware, where the testing is done using the trained models. The result shows that the validation accuracy obtained in the case of the eye dataset is around 90% whereas in the case of lung dataset it is around 63%. The proposed system aims to help medical professionals to provide a diagnosis with better accuracy thus helping in reducing infant mortality due to pneumonia and allowing finding the severity of eye disease at an earlier stage.

scholarly journals Detection of COVID-19 and Other Pneumonia Cases Using Convolutional Neural Networks and X-ray Images

2021 ◽  
Vol 42 (1) ◽  
pp. e90289
Author(s):  
Carlos Eduardo Belman López
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Classification Accuracy ◽  
Network Capacity ◽  
Superior Performance ◽  
Data Set ◽  
X Ray ◽  
Regularization Techniques ◽  
The Impact

Given that it is fundamental to detect positive COVID-19 cases and treat affected patients quickly to mitigate the impact of the virus, X-ray images have been subjected to research regarding COVID-19, together with deep learning models, eliminating disadvantages such as the scarcity of RT-PCR test kits, their elevated costs, and the long wait for results. The contribution of this paper is to present new models for detecting COVID-19 and other cases of pneumonia using chest X-ray images and convolutional neural networks, thus providing accurate diagnostics in binary and 4-classes classification scenarios. Classification accuracy was improved, and overfitting was prevented by following 2 actions: (1) increasing the data set size while the classification scenarios were balanced; and (2) adding regularization techniques and performing hyperparameter optimization. Additionally, the network capacity and size in the models were reduced as much as possible, making the final models a perfect option to be deployed locally on devices with limited capacities and without the need for Internet access. The impact of key hyperparameters was tested using modern deep learning packages. The final models obtained a classification accuracy of 99,17 and 94,03% for the binary and categorical scenarios, respectively, achieving superior performance compared to other studies in the literature, and requiring a significantly lower number of parameters. The models can also be placed on a digital platform to provide instantaneous diagnostics and surpass the shortage of experts and radiologists.

Sign in / Sign up

Export Citation Format

Share Document