Latency Estimation Tool and Investigation of Neural Networks Inference on Mobile GPU

A lot of deep learning applications are desired to be run on mobile devices. Both accuracy and inference time are meaningful for a lot of them. While the number of FLOPs is usually used as a proxy for neural network latency, it may not be the best choice. In order to obtain a better approximation of latency, the research community uses lookup tables of all possible layers for the calculation of the inference on a mobile CPU. It requires only a small number of experiments. Unfortunately, on a mobile GPU, this method is not applicable in a straightforward way and shows low precision. In this work, we consider latency approximation on a mobile GPU as a data- and hardware-specific problem. Our main goal is to construct a convenient Latency Estimation Tool for Investigation (LETI) of neural network inference and building robust and accurate latency prediction models for each specific task. To achieve this goal, we make tools that provide a convenient way to conduct massive experiments on different target devices focusing on a mobile GPU. After evaluation of the dataset, one can train the regression model on experimental data and use it for future latency prediction and analysis. We experimentally demonstrate the applicability of such an approach on a subset of the popular NAS-Benchmark 101 dataset for two different mobile GPU.

Download Full-text

Learners Demographics Classification on MOOCs During the COVID-19: Author Profiling via Deep Learning Based on Semantic and Syntactic Representations

Frontiers in Research Metrics and Analytics ◽

10.3389/frma.2021.673928 ◽

2021 ◽

Vol 6 ◽

Author(s):

Tahani Aljohani ◽

Alexandra I. Cristea

Keyword(s):

Neural Network ◽

Neural Networks ◽

Deep Learning ◽

Prediction Models ◽

Short Term Memory ◽

Methodological Approach ◽

High Accuracy ◽

Directional Model ◽

Textual Representations ◽

The One

Massive Open Online Courses (MOOCs) have become universal learning resources, and the COVID-19 pandemic is rendering these platforms even more necessary. In this paper, we seek to improve Learner Profiling (LP), i.e. estimating the demographic characteristics of learners in MOOC platforms. We have focused on examining models which show promise elsewhere, but were never examined in the LP area (deep learning models) based on effective textual representations. As LP characteristics, we predict here the employment status of learners. We compare sequential and parallel ensemble deep learning architectures based on Convolutional Neural Networks and Recurrent Neural Networks, obtaining an average high accuracy of 96.3% for our best method. Next, we predict the gender of learners based on syntactic knowledge from the text. We compare different tree-structured Long-Short-Term Memory models (as state-of-the-art candidates) and provide our novel version of a Bi-directional composition function for existing architectures. In addition, we evaluate 18 different combinations of word-level encoding and sentence-level encoding functions. Based on these results, we show that our Bi-directional model outperforms all other models and the highest accuracy result among our models is the one based on the combination of FeedForward Neural Network and the Stack-augmented Parser-Interpreter Neural Network (82.60% prediction accuracy). We argue that our prediction models recommended for both demographics characteristics examined in this study can achieve high accuracy. This is additionally also the first time a sound methodological approach toward improving accuracy for learner demographics classification on MOOCs was proposed.

Download Full-text

Estimating latent positions of actors using Neural Networks in R with GCN4R

10.1101/2020.11.02.364935 ◽

2020 ◽

Author(s):

Joshua Levy ◽

Carly Bobak ◽

Brock Christensen ◽

Louis Vaickus ◽

James O’Malley

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Learning Community ◽

Biological Networks ◽

Peer Effects ◽

Prediction Models ◽

Research Community ◽

Social Network Data ◽

Covariate Information ◽

Graph Neural Networks

AbstractNetwork analysis methods are useful to better understand and contextualize relationships between entities. While statistical and machine learning prediction models generally assume independence between actors, network-based statistical methods for social network data allow for dyadic dependence between actors. While numerous methods have been developed for the R statistical software to analyze such data, deep learning methods have not been implemented in this language. Here, we introduce GCN4R, an R library for fitting graph neural networks on independent networks to aggregate actor covariate information to yield meaningful embeddings for a variety of network-based tasks (e.g. community detection, peer effects models, social influence). We provide an extensive overview of insights and methods utilized by the deep learning community on learning on social and biological networks, followed by a tutorial that demonstrates some of the capabilities of the GCN4R framework to make these methods more accessible to the R research community.

Download Full-text

GMDH-type Neural Networks for Predicting Financial Time Series: A Study of Informational Efficiency of Stock Markets

10.21203/rs.3.rs-1017105/v1 ◽

2021 ◽

Author(s):

L Jakaite ◽

M Ciemny ◽

S Selitskiy ◽

Vitaly Schetinin

Keyword(s):

Neural Network ◽

Neural Networks ◽

Deep Learning ◽

Prediction Models ◽

Network Models ◽

Efficient Market Hypothesis ◽

Group Method ◽

Informational Efficiency ◽

Prediction Errors ◽

Neural Network Models

Abstract A theory of Efficient Market Hypothesis (EMH) has been introduced by Fama to analyse financial markets. In particular the EMH theory has been proven in real cases under different conditions, including financial crises and frauds. The EMH assumes to examine the prediction accuracy of models designed on retrospective data. Such prediction models could be designed in different ways that motivated us to explore Machine Learning (ML) methods known for building models providing a high prediction performance. In this study we propose a ``deep'' learning method for building high-performance prediction models. The proposed method is based on the Group Method of Data Handling (GMDH) that is the deep learning paradigm capable of building multilayer neural-network models of a near-optimal complexity on given data. We show that the developed GMDH-type neural network has outperformed the models built by the conventional ML methods on the Warsaw Stock Exchange data. It is important that the complexity of the designed GMDH-type neural-networks is defined by the number of layers and connections between neurons. The performances of models were compared in terms of the prediction errors. We report a significantly smaller prediction error of the proposed method than that of the conventional autoregressive and "shallow’’ neural-network models. This finally allows us to conclude that traders will be advantaged by the proposed method.

Download Full-text

Automatic Defects Segmentation and Identification by Deep Learning Algorithm with Pulsed Thermography: Synthetic and Experimental Data

Big Data and Cognitive Computing ◽

10.3390/bdcc5010009 ◽

2021 ◽

Vol 5 (1) ◽

pp. 9

Author(s):

Qiang Fang ◽

Clemente Ibarra-Castanedo ◽

Xavier Maldague

Keyword(s):

Neural Network ◽

Experimental Data ◽

Neural Networks ◽

Deep Learning ◽

Infrared Thermography ◽

Learning Algorithm ◽

Synthetic Data ◽

Training Data ◽

Experimental Database ◽

The Neural Network

In quality evaluation (QE) of the industrial production field, infrared thermography (IRT) is one of the most crucial techniques used for evaluating composite materials due to the properties of low cost, fast inspection of large surfaces, and safety. The application of deep neural networks tends to be a prominent direction in IRT Non-Destructive Testing (NDT). During the training of the neural network, the Achilles heel is the necessity of a large database. The collection of huge amounts of training data is the high expense task. In NDT with deep learning, synthetic data contributing to training in infrared thermography remains relatively unexplored. In this paper, synthetic data from the standard Finite Element Models are combined with experimental data to build repositories with Mask Region based Convolutional Neural Networks (Mask-RCNN) to strengthen the neural network, learning the essential features of objects of interest and achieving defect segmentation automatically. These results indicate the possibility of adapting inexpensive synthetic data merging with a certain amount of the experimental database for training the neural networks in order to achieve the compelling performance from a limited collection of the annotated experimental data of a real-world practical thermography experiment.

Download Full-text

Convolutional Neural Network for the Semantic Segmentation of Remote Sensing Images

Mobile Networks and Applications ◽

10.1007/s11036-020-01703-3 ◽

2021 ◽

Vol 26 (1) ◽

pp. 200-215

Author(s):

Muhammad Alam ◽

Jian-Feng Wang ◽

Cong Guangpei ◽

LV Yunrong ◽

Yuanfang Chen

Keyword(s):

Neural Network ◽

Remote Sensing ◽

Neural Networks ◽

Image Processing ◽

Deep Learning ◽

Semantic Segmentation ◽

Natural Scene ◽

Remote Sensing Images ◽

Advantages And Disadvantages ◽

Target Segmentation

AbstractIn recent years, the success of deep learning in natural scene image processing boosted its application in the analysis of remote sensing images. In this paper, we applied Convolutional Neural Networks (CNN) on the semantic segmentation of remote sensing images. We improve the Encoder- Decoder CNN structure SegNet with index pooling and U-net to make them suitable for multi-targets semantic segmentation of remote sensing images. The results show that these two models have their own advantages and disadvantages on the segmentation of different objects. In addition, we propose an integrated algorithm that integrates these two models. Experimental results show that the presented integrated algorithm can exploite the advantages of both the models for multi-target segmentation and achieve a better segmentation compared to these two models.

Download Full-text

Financial Information Asymmetry: Using Deep Learning Algorithms to Predict Financial Distress

Symmetry ◽

10.3390/sym13030443 ◽

2021 ◽

Vol 13 (3) ◽

pp. 443

Author(s):

Chyan-long Jan

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Information Asymmetry ◽

Error Rate ◽

Financial Distress ◽

Prediction Models ◽

High Accuracy ◽

Financial Information ◽

Financial Distress Prediction ◽

Distress Prediction

Because of the financial information asymmetry, the stakeholders usually do not know a company’s real financial condition until financial distress occurs. Financial distress not only influences a company’s operational sustainability and damages the rights and interests of its stakeholders, it may also harm the national economy and society; hence, it is very important to build high-accuracy financial distress prediction models. The purpose of this study is to build high-accuracy and effective financial distress prediction models by two representative deep learning algorithms: Deep neural networks (DNN) and convolutional neural networks (CNN). In addition, important variables are selected by the chi-squared automatic interaction detector (CHAID). In this study, the data of Taiwan’s listed and OTC sample companies are taken from the Taiwan Economic Journal (TEJ) database during the period from 2000 to 2019, including 86 companies in financial distress and 258 not in financial distress, for a total of 344 companies. According to the empirical results, with the important variables selected by CHAID and modeling by CNN, the CHAID-CNN model has the highest financial distress prediction accuracy rate of 94.23%, and the lowest type I error rate and type II error rate, which are 0.96% and 4.81%, respectively.

Download Full-text

Enabling deeper learning on big data for materials informatics applications

Scientific Reports ◽

10.1038/s41598-021-83193-1 ◽

2021 ◽

Vol 11 (1) ◽

Author(s):

Dipendra Jha ◽

Vishu Gupta ◽

Logan Ward ◽

Zijiang Yang ◽

Christopher Wolverton ◽

...

Keyword(s):

Neural Networks ◽

Big Data ◽

Deep Learning ◽

Deep Neural Networks ◽

Materials Science ◽

Prediction Models ◽

Model Performance ◽

Materials Informatics ◽

Learning Framework ◽

Significant Attention

AbstractThe application of machine learning (ML) techniques in materials science has attracted significant attention in recent years, due to their impressive ability to efficiently extract data-driven linkages from various input materials representations to their output properties. While the application of traditional ML techniques has become quite ubiquitous, there have been limited applications of more advanced deep learning (DL) techniques, primarily because big materials datasets are relatively rare. Given the demonstrated potential and advantages of DL and the increasing availability of big materials datasets, it is attractive to go for deeper neural networks in a bid to boost model performance, but in reality, it leads to performance degradation due to the vanishing gradient problem. In this paper, we address the question of how to enable deeper learning for cases where big materials data is available. Here, we present a general deep learning framework based on Individual Residual learning (IRNet) composed of very deep neural networks that can work with any vector-based materials representation as input to build accurate property prediction models. We find that the proposed IRNet models can not only successfully alleviate the vanishing gradient problem and enable deeper learning, but also lead to significantly (up to 47%) better model accuracy as compared to plain deep neural networks and traditional ML techniques for a given input materials representation in the presence of big data.

Download Full-text

Beyond Deep Learning: An Econometric Example

International Journal of Uncertainty Fuzziness and Knowledge-Based Systems ◽

10.1142/s0218488520400036 ◽

2020 ◽

Vol 28 (Supp01) ◽

pp. 31-38 ◽

Cited By ~ 1

Author(s):

Ruofan Liao ◽

Paravee Maneejuk ◽

Songsak Sriboonchitta

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Prediction Models ◽

Original Data ◽

Parametric Model ◽

Parametric Models ◽

The Past ◽

Currency Exchange ◽

Currency Exchange Rate ◽

Linear And Nonlinear

In the past, in many areas, the best prediction models were linear and nonlinear parametric models. In the last decade, in many application areas, deep learning has shown to lead to more accurate predictions than the parametric models. Deep learning-based predictions are reasonably accurate, but not perfect. How can we achieve better accuracy? To achieve this objective, we propose to combine neural networks with parametric model: namely, to train neural networks not on the original data, but on the differences between the actual data and the predictions of the parametric model. On the example of predicting currency exchange rate, we show that this idea indeed leads to more accurate predictions.

Download Full-text

Classification of Skin Disease Using Deep Learning Neural Networks with MobileNet V2 and LSTM

Sensors ◽

10.3390/s21082852 ◽

2021 ◽

Vol 21 (8) ◽

pp. 2852

Author(s):

Parvathaneni Naga Srinivasu ◽

Jalluri Gnana SivaSai ◽

Muhammad Fazal Ijaz ◽

Akash Kumar Bhoi ◽

Wonjoon Kim ◽

...

Keyword(s):

Neural Network ◽

Neural Networks ◽

Deep Learning ◽

Convolutional Neural Network ◽

Skin Disease ◽

Network Architecture ◽

Large Scale ◽

Short Term Memory ◽

Convolutional Networks ◽

Occurrence Matrix

Deep learning models are efficient in learning the features that assist in understanding complex patterns precisely. This study proposed a computerized process of classifying skin disease through deep learning based MobileNet V2 and Long Short Term Memory (LSTM). The MobileNet V2 model proved to be efficient with a better accuracy that can work on lightweight computational devices. The proposed model is efficient in maintaining stateful information for precise predictions. A grey-level co-occurrence matrix is used for assessing the progress of diseased growth. The performance has been compared against other state-of-the-art models such as Fine-Tuned Neural Networks (FTNN), Convolutional Neural Network (CNN), Very Deep Convolutional Networks for Large-Scale Image Recognition developed by Visual Geometry Group (VGG), and convolutional neural network architecture that expanded with few changes. The HAM10000 dataset is used and the proposed method has outperformed other methods with more than 85% accuracy. Its robustness in recognizing the affected region much faster with almost 2× lesser computations than the conventional MobileNet model results in minimal computational efforts. Furthermore, a mobile application is designed for instant and proper action. It helps the patient and dermatologists identify the type of disease from the affected region’s image at the initial stage of the skin disease. These findings suggest that the proposed system can help general practitioners efficiently and effectively diagnose skin conditions, thereby reducing further complications and morbidity.

Download Full-text

Abstract P437: Deep Learning-based Models For Complete Atrioventricular Block Heart Rhythm Analysis

Circulation Research ◽

10.1161/res.129.suppl_1.p437 ◽

2021 ◽

Vol 129 (Suppl_1) ◽

Author(s):

Dahim Choi ◽

Nam Kyun Kim ◽

Young H Son ◽

Yuming Gao ◽

Christina Sheng ◽

...

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Atrioventricular Block ◽

Prediction Models ◽

Pacemaker Implantation ◽

Heart Rhythm ◽

High Sensitivity ◽

Ventricular Myocardium ◽

True Positive Rate ◽

Starting Point

Atrioventricular block (AVB), caused by impairment in the heart conduction system, presents extreme diversity and is associated with other complications. Only half of AVB patients require a permanent pacemaker, and the process determining the pacemaker implantation is associated with an increase in cost and patient morbidity and mortality. Thus, there is a need for models capable of accurately identifying transient or reversible causes for conduction disturbances and predicting the patient risks and the necessity of a pacemaker. Deep learning (DL) is brought to the forefront due to its prediction accuracy, and the DL-based electrocardiogram (ECG) analysis can be a breakthrough to analyze a massive amount of data. However, the current DL models are unsuitable for AVB-ECG, where the P waves are decoupled from the QRS/T waves, and a black-box nature of the DL-based model lowers the credibility of prediction models to physicians. Here, we present a real-time-capable DL-based algorithm that can identify AVB-ECG waves and automate AVB phenotyping for arrhythmogenic risk assessment. Our algorithm can analyze unformatted ECG records with abnormal patterns by integrating the two representative DL algorithms: convolutional neural networks (CNN) and recurrent neural networks (RNN). This hybrid CNN/RNN network can memorize local patterns, spatial hierarchies, and long-range temporal dependencies of ECG signals. Furthermore, by integrating parameters derived from dimension reduction analysis and heart rate variability into the hybrid layers, the algorithm can capture the P/QRS/T-specific morphological and temporal features in ECG waveforms. We evaluated the algorithm using the six AVB porcine models, where TBX18, a pacemaker transcription factor, was transduced into the ventricular myocardium to form a biological pacemaker, and an additional electronic pacemaker was transplanted as a backup pacemaker. We achieved high sensitivity (95% true positive rate) and quantified the potential risks of various pathological ECG patterns. This study may be a starting point in conducting both retrospective and prospective patient studies and will help physicians understand its decision-making workflow and find the incorrect recommendations for AVB patients.

Download Full-text