scholarly journals DLBench: a comprehensive experimental evaluation of deep learning frameworks

2021 ◽  
Author(s):  
Radwa Elshawi ◽  
Abdul Wahab ◽  
Ahmed Barnawi ◽  
Sherif Sakr
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Language Processing ◽  
Experimental Evaluation ◽  
Short Term Memory ◽  
Performance Characteristics ◽  
Training Time ◽  
Learning Frameworks ◽  
Learning Architectures

AbstractDeep Learning (DL) has achieved remarkable progress over the last decade on various tasks such as image recognition, speech recognition, and natural language processing. In general, three main crucial aspects fueled this progress: the increasing availability of large amount of digitized data, the increasing availability of affordable parallel and powerful computing resources (e.g., GPU) and the growing number of open source deep learning frameworks that facilitate and ease the development process of deep learning architectures. In practice, the increasing popularity of deep learning frameworks calls for benchmarking studies that can effectively evaluate and understand the performance characteristics of these systems. In this paper, we conduct an extensive experimental evaluation and analysis of six popular deep learning frameworks, namely, TensorFlow, MXNet, PyTorch, Theano, Chainer, and Keras, using three types of DL architectures Convolutional Neural Networks (CNN), Faster Region-based Convolutional Neural Networks (Faster R-CNN), and Long Short Term Memory (LSTM). Our experimental evaluation considers different aspects for its comparison including accuracy, training time, convergence and resource consumption patterns. Our experiments have been conducted on both CPU and GPU environments using different datasets. We report and analyze the performance characteristics of the studied frameworks. In addition, we report a set of insights and important lessons that we have learned from conducting our experiments.

scholarly journals Stock Pattern Classification from Charts using Deep Learning Algorithms

2020 ◽  
Vol 3 (1) ◽  
pp. 445-454
Author(s):  
Celal Buğra Kaya ◽  
Alperen Yılmaz ◽  
Gizem Nur Uzun ◽  
Zeynep Hilal Kilimci
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Pattern Classification ◽  
Short Term Memory ◽  
Stock Exchange ◽  
Learning Techniques ◽  
Proposed Model ◽  
Istanbul Stock Exchange

Pattern classification is related with the automatic finding of regularities in dataset through the utilization of various learning techniques. Thus, the classification of the objects into a set of categories or classes is provided. This study is undertaken to evaluate deep learning methodologies to the classification of stock patterns. In order to classify patterns that are obtained from stock charts, convolutional neural networks (CNNs), recurrent neural networks (RNNs), and long-short term memory networks (LSTMs) are employed. To demonstrate the efficiency of proposed model in categorizing patterns, hand-crafted image dataset is constructed from stock charts in Istanbul Stock Exchange and NASDAQ Stock Exchange. Experimental results show that the usage of convolutional neural networks exhibits superior classification success in recognizing patterns compared to the other deep learning methodologies.

scholarly journals Newspaper Article Classification using Machine Learning Techniques

2020 ◽  
Vol 9 (5) ◽  
pp. 872-877
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Machine Learning Techniques ◽  
Learning Approach ◽  
Training Time ◽  
Time Prediction ◽  
Learning Techniques ◽  
Prediction Time ◽  
Newspaper Articles

Newspaper articles offer us insights on several news. They can be one of many categories like sports, politics, Science and Technology etc. Text classification is a need of the day as large uncategorized data is the problem everywhere. Through this study, We intend to compare several algorithms along with data preprocessing approaches to classify the newspaper articles into their respective categories. Convolutional Neural Networks(CNN) is a deep learning approach which is currently a strong competitor to other classification algorithms like SVM, Naive Bayes and KNN. We hence intend to implement Convolutional Neural Networks - a deep learning approach to classify our newspaper articles, develop an understanding of all the algorithms implemented and compare their results. We also attempt to compare the training time, prediction time and accuracies of all the algorithms.

scholarly journals A comparative analysis on question classification task based on deep learning approaches

2021 ◽  
Vol 7 ◽  
pp. e570
Author(s):  
Muhammad Zulqarnain ◽  
Ahmed Khalaf Zager Alsaedi ◽  
Rozaida Ghazali ◽  
Muhammad Ghulam Ghouse ◽  
Wareesa Sharif ◽  
...  
Keyword(s):  
Deep Learning ◽  
Language Processing ◽  
Web Mining ◽  
Question Answering ◽  
Short Term Memory ◽  
Learning Approaches ◽  
Question Classification ◽  
Considerable Impact ◽  
Turkish Language ◽  
Learning Architectures

Question classification is one of the essential tasks for automatic question answering implementation in natural language processing (NLP). Recently, there have been several text-mining issues such as text classification, document categorization, web mining, sentiment analysis, and spam filtering that have been successfully achieved by deep learning approaches. In this study, we illustrated and investigated our work on certain deep learning approaches for question classification tasks in an extremely inflected Turkish language. In this study, we trained and tested the deep learning architectures on the questions dataset in Turkish. In addition to this, we used three main deep learning approaches (Gated Recurrent Unit (GRU), Long Short-Term Memory (LSTM), Convolutional Neural Networks (CNN)) and we also applied two different deep learning combinations of CNN-GRU and CNN-LSTM architectures. Furthermore, we applied the Word2vec technique with both skip-gram and CBOW methods for word embedding with various vector sizes on a large corpus composed of user questions. By comparing analysis, we conducted an experiment on deep learning architectures based on test and 10-cross fold validation accuracy. Experiment results were obtained to illustrate the effectiveness of various Word2vec techniques that have a considerable impact on the accuracy rate using different deep learning approaches. We attained an accuracy of 93.7% by using these techniques on the question dataset.

scholarly journals Image Classification: A Survey

2020 ◽  
Vol 1 (2) ◽  
pp. 1-9
Author(s):  
Ankita Singh ◽  
◽  
Pawan Singh
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Image Classification ◽  
Convolutional Neural Networks ◽  
Language Processing ◽  
Classification Accuracy ◽  
Deep Neural Network ◽  
Learning Ability ◽  
Final Decision

The Classification of images is a paramount topic in artificial vision systems which have drawn a notable amount of interest over the past years. This field aims to classify an image, which is an input, based on its visual content. Currently, most people relied on hand-crafted features to describe an image in a particular way. Then, using classifiers that are learnable, such as random forest, and decision tree was applied to the extract features to come to a final decision. The problem arises when large numbers of photos are concerned. It becomes a too difficult problem to find features from them. This is one of the reasons that the deep neural network model has been introduced. Owing to the existence of Deep learning, it can become feasible to represent the hierarchical nature of features using a various number of layers and corresponding weight with them. The existing image classification methods have been gradually applied in real-world problems, but then there are various problems in its application processes, such as unsatisfactory effect and extremely low classification accuracy or then and weak adaptive ability. Models using deep learning concepts have robust learning ability, which combines the feature extraction and the process of classification into a whole which then completes an image classification task, which can improve the image classification accuracy effectively. Convolutional Neural Networks are a powerful deep neural network technique. These networks preserve the spatial structure of a problem and were built for object recognition tasks such as classifying an image into respective classes. Neural networks are much known because people are getting a state-of-the-art outcome on complex computer vision and natural language processing tasks. Convolutional neural networks have been extensively used.

scholarly journals A Study of The Convolutional Neural Networks Applications

2019 ◽  
Vol 3 (2) ◽  
pp. 31-40 ◽  
Author(s):  
Ahmed Shamsaldin ◽  
Polla Fattah ◽  
Tarik Rashid ◽  
Nawzad Al-Salihi
Keyword(s):  
Neural Networks ◽  
Computer Vision ◽  
Deep Learning ◽  
Natural Language Processing ◽  
Face Recognition ◽  
Convolutional Neural Networks ◽  
Language Processing ◽  
Text Classification ◽  
Scene Labeling ◽  
Real World Problems

At present, deep learning is widely used in a broad range of arenas. A convolutional neural networks (CNN) is becoming the star of deep learning as it gives the best and most precise results when cracking real-world problems. In this work, a brief description of the applications of CNNs in two areas will be presented: First, in computer vision, generally, that is, scene labeling, face recognition, action recognition, and image classification; Second, in natural language processing, that is, the fields of speech recognition and text classification.

scholarly journals Telugu and Hindi Script Recognition using Deep learning Techniques

2019 ◽  
Vol 8 (11) ◽  
pp. 1758-1764 ◽  
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Character Recognition ◽  
Research Work ◽  
Indian Languages ◽  
Digital Form ◽  
Training Time ◽  
Learning Techniques ◽  
Handwritten Document

The need for offline handwritten character recognition is intense, yet difficult as the writing varies from person to person and also depends on various other factors connected to the attitude and mood of the person. However, we are able to achieve it by converting the handwritten document into digital form. It has been advanced with introducing convolutional neural networks and is further productive with pre-trained models which have the capacity of decreasing the training time and increasing accuracy of character recognition. Research in recognition of handwritten characters for Indian languages is less when compared to other languages like English, Latin, Chinese etc., mainly because it is a multilingual country. Recognition of Telugu and Hindi characters are more difficult as the script of these languages is mostly cursive and are with more diacritics. So the research work in this line is to have inclination towards accuracy in their recognition. Some research has already been started and is successful up to eighty percent in offline hand written character recognition of Telugu and Hindi. The proposed work focuses on increasing accuracy in less time in recognition of these selected languages and is able to reach the expectant values.

scholarly journals AN APPROACH TO ACCELERATE THE TRAINING OF CONVOLUTIONAL NEURAL NETWORKS BY TUNING THE HYPERPARAMETERS OF LEARNING

2020 ◽  
Vol 2 (2) ◽  
pp. 32-37
Author(s):  
P. RADIUK ◽  
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Video Processing ◽  
Network Architecture ◽  
Random Search ◽  
Machine Learning Algorithms ◽  
Language Recognition ◽  
Data Set ◽  
Training Time

Over the last decade, a set of machine learning algorithms called deep learning has led to significant improvements in computer vision, natural language recognition and processing. This has led to the widespread use of a variety of commercial, learning-based products in various fields of human activity. Despite this success, the use of deep neural networks remains a black box. Today, the process of setting hyperparameters and designing a network architecture requires experience and a lot of trial and error and is based more on chance than on a scientific approach. At the same time, the task of simplifying deep learning is extremely urgent. To date, no simple ways have been invented to establish the optimal values of learning hyperparameters, namely learning speed, sample size, data set, learning pulse, and weight loss. Grid search and random search of hyperparameter space are extremely resource intensive. The choice of hyperparameters is critical for the training time and the final result. In addition, experts often choose one of the standard architectures (for example, ResNets and ready-made sets of hyperparameters. However, such kits are usually suboptimal for specific practical tasks. The presented work offers an approach to finding the optimal set of hyperparameters of learning ZNM. An integrated approach to all hyperparameters is valuable because there is an interdependence between them. The aim of the work is to develop an approach for setting a set of hyperparameters, which will reduce the time spent during the design of ZNM and ensure the efficiency of its work. In recent decades, the introduction of deep learning methods, in particular convolutional neural networks (CNNs), has led to impressive success in image and video processing. However, the training of CNN has been commonly mostly based on the employment of quasi-optimal hyperparameters. Such an approach usually requires huge computational and time costs to train the network and does not guarantee a satisfactory result. However, hyperparameters play a crucial role in the effectiveness of CNN, as diverse hyperparameters lead to models with significantly different characteristics. Poorly selected hyperparameters generally lead to low model performance. The issue of choosing optimal hyperparameters for CNN has not been resolved yet. The presented work proposes several practical approaches to setting hyperparameters, which allows reducing training time and increasing the accuracy of the model. The article considers the function of training validation loss during underfitting and overfitting. There are guidelines in the end to reach the optimization point. The paper also considers the regulation of learning rate and momentum to accelerate network training. All experiments are based on the widespread CIFAR-10 and CIFAR-100 datasets.

Exemplar models are useful and deep neural networks overcome their limitations: A commentary on Ambridge (2020)

2020 ◽  
Vol 40 (5-6) ◽  
pp. 612-615
Author(s):  
James L. McClelland
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Language Processing ◽  
Deep Neural Networks ◽  
Machine Intelligence ◽  
Human Language ◽  
Language Usage ◽  
Primitive Elements ◽  
Exemplar Models ◽  
Learning Architectures

Humans are sensitive to the properties of individual items, and exemplar models are useful for capturing this sensitivity. I am a proponent of an extension of exemplar-based architectures that I briefly describe. However, exemplar models are very shallow architectures in which it is necessary to stipulate a set of primitive elements that make up each example, and such architectures have not been as successful as deep neural networks in capturing language usage and meaning. More work is needed bringing contemporary deep learning architectures used in machine intelligence to the effort to understand human language processing.

scholarly journals Pneumonia and Eye Disease Detection using Convolutional Neural Networks

2020 ◽  
Vol 10 (3) ◽  
pp. 5769-5774 ◽  
Author(s):  
P. Chakraborty ◽  
C. Tharini
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Detection System ◽  
Eye Disease ◽  
Medical Professionals ◽  
Disease Detection ◽  
Training Time ◽  
X Ray ◽  
Detection Systems

Automatic disease detection systems based on Convolutional Neural Networks (CNNs) are proposed in this paper for helping the medical professionals in the detection of diseases from scan and X-ray images. CNN based classification helps decision making in a prompt manner with high precision. CNNs are a subset of deep learning which is a branch of Artificial Intelligence. The main advantage of CNNs compared to other deep learning algorithms is that they require minimal pre-processing. In the proposed disease detection system, two medical image datasets consisting of Optical Coherence Tomography (OCT) and chest X-ray images of 1-5 year-old children are considered and used as inputs. The medical images are processed and classified using CNN and various performance measuring parameters such as accuracy, loss, and training time are measured. The system is then implemented in hardware, where the testing is done using the trained models. The result shows that the validation accuracy obtained in the case of the eye dataset is around 90% whereas in the case of lung dataset it is around 63%. The proposed system aims to help medical professionals to provide a diagnosis with better accuracy thus helping in reducing infant mortality due to pneumonia and allowing finding the severity of eye disease at an earlier stage.

scholarly journals Alzheimer's Disease Classification Using 2D Convolutional Neural Networks

2021 ◽  
Author(s):  
Xin Xing ◽  
Liangliang Liu ◽  
Qi Yin ◽  
Gongbo Liang
Keyword(s):  
Alzheimer’S Disease ◽  
Neural Networks ◽  
Alzheimer's Disease ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Imaging Modality ◽  
Model Performance ◽  
Brain Magnetic Resonance Imaging ◽  
Disease Classification ◽  
Training Time

Alzheimer's disease (AD) is a non-treatable and non-reversible disease that affects about 6% of people who are 65 and older. Brain magnetic resonance imaging (MRI) is a pseudo-3D imaging modality that is widely used for AD diagnosis. Convolutional neural networks with 3D kernels (3D CNNs) are often the default choice for deep learning based MRI analysis. However, 3D CNNs are usually computationally costly and data-hungry. Such disadvantages post a barrier of using modern deep learning techniques in the medical imaging domain, in which the number of data can be used for training is usually limited. In this work, we propose three approaches that leverage 2D CNNs on 3D MRI data. We test the proposed methods on the Alzheimer's Disease Neuroimaging Initiative dataset across two popular 2D CNN architectures. The evaluation results show that the proposed method improves the model performance on AD diagnosis by 8.33% accuracy or 10.11% auROC, while significantly reduce the training time by over 89%. We also discuss the potential causes for performance improvement and the limitation. We believe this work can serve as a strong baseline for future researchers.

Sign in / Sign up

Export Citation Format

Share Document