Beyond Deep Learning: An Econometric Example

2020 ◽  
Vol 28 (Supp01) ◽  
pp. 31-38 ◽  
Author(s):  
Ruofan Liao ◽  
Paravee Maneejuk ◽  
Songsak Sriboonchitta
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Prediction Models ◽  
Original Data ◽  
Parametric Model ◽  
Parametric Models ◽  
The Past ◽  
Currency Exchange ◽  
Currency Exchange Rate ◽  
Linear And Nonlinear

In the past, in many areas, the best prediction models were linear and nonlinear parametric models. In the last decade, in many application areas, deep learning has shown to lead to more accurate predictions than the parametric models. Deep learning-based predictions are reasonably accurate, but not perfect. How can we achieve better accuracy? To achieve this objective, we propose to combine neural networks with parametric model: namely, to train neural networks not on the original data, but on the differences between the actual data and the predictions of the parametric model. On the example of predicting currency exchange rate, we show that this idea indeed leads to more accurate predictions.

scholarly journals The Dependence of the CZK Exchange Rate on the Development of the GDP in the Czech Republic in the Conditions of an Expected Crisis

2021 ◽  
Vol 92 ◽  
pp. 08013
Author(s):  
Veronika Machová ◽  
Tomáš Krulický
Keyword(s):  
Neural Networks ◽  
Exchange Rate ◽  
Czech Republic ◽  
Value Added ◽  
The Czech Republic ◽  
The Past ◽  
Currency Exchange ◽  
Currency Exchange Rate ◽  
The World ◽  
Rate Development

Research background: In the past, some studies proved that the development of a currency exchange rate predicts the development of the whole national economy. The monetary market overtakes the development of the actual economy for a few months. Does this apply also in the case of the Czech koruna, in the era of the global Coronavirus pandemics and in the world affected by the pandemics? Purpose of the article: The main objective is to analyze a dependence of the Czech koruna (CZK) to Euro (EUR) exchange rate development on gross domestic product of the Czech Republic in the conditions of an expected crisis. Methods: The data used of the analysis are represented by the information about the CZK and EUR exchange rate from the beginning of 1999 to the 15th June 2020 and by the quarterly development of the Czech GDP. To measure the dependence and predict the development of the GDP based on the CZK exchange rate development, the method of AI is used, namely the regression analysis using the artificial neural networks. Findings & Value added: The effect of EUR/CZK on GDP can be quantified reaching around 31%. It is assumed that the GDP will fall significantly in 2020 with a certain growth only being possibly expected in 2021 (even more significantly in the second quarter of 2021). Due to the GDP development, the development of the EUR/CZK could then be forecasted as well.

scholarly journals Financial Information Asymmetry: Using Deep Learning Algorithms to Predict Financial Distress

Symmetry ◽  
2021 ◽  
Vol 13 (3) ◽  
pp. 443
Author(s):  
Chyan-long Jan
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Information Asymmetry ◽  
Error Rate ◽  
Financial Distress ◽  
Prediction Models ◽  
High Accuracy ◽  
Financial Information ◽  
Financial Distress Prediction ◽  
Distress Prediction

Because of the financial information asymmetry, the stakeholders usually do not know a company’s real financial condition until financial distress occurs. Financial distress not only influences a company’s operational sustainability and damages the rights and interests of its stakeholders, it may also harm the national economy and society; hence, it is very important to build high-accuracy financial distress prediction models. The purpose of this study is to build high-accuracy and effective financial distress prediction models by two representative deep learning algorithms: Deep neural networks (DNN) and convolutional neural networks (CNN). In addition, important variables are selected by the chi-squared automatic interaction detector (CHAID). In this study, the data of Taiwan’s listed and OTC sample companies are taken from the Taiwan Economic Journal (TEJ) database during the period from 2000 to 2019, including 86 companies in financial distress and 258 not in financial distress, for a total of 344 companies. According to the empirical results, with the important variables selected by CHAID and modeling by CNN, the CHAID-CNN model has the highest financial distress prediction accuracy rate of 94.23%, and the lowest type I error rate and type II error rate, which are 0.96% and 4.81%, respectively.

scholarly journals Enabling deeper learning on big data for materials informatics applications

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Dipendra Jha ◽  
Vishu Gupta ◽  
Logan Ward ◽  
Zijiang Yang ◽  
Christopher Wolverton ◽  
...  
Keyword(s):  
Neural Networks ◽  
Big Data ◽  
Deep Learning ◽  
Deep Neural Networks ◽  
Materials Science ◽  
Prediction Models ◽  
Model Performance ◽  
Materials Informatics ◽  
Learning Framework ◽  
Significant Attention

AbstractThe application of machine learning (ML) techniques in materials science has attracted significant attention in recent years, due to their impressive ability to efficiently extract data-driven linkages from various input materials representations to their output properties. While the application of traditional ML techniques has become quite ubiquitous, there have been limited applications of more advanced deep learning (DL) techniques, primarily because big materials datasets are relatively rare. Given the demonstrated potential and advantages of DL and the increasing availability of big materials datasets, it is attractive to go for deeper neural networks in a bid to boost model performance, but in reality, it leads to performance degradation due to the vanishing gradient problem. In this paper, we address the question of how to enable deeper learning for cases where big materials data is available. Here, we present a general deep learning framework based on Individual Residual learning (IRNet) composed of very deep neural networks that can work with any vector-based materials representation as input to build accurate property prediction models. We find that the proposed IRNet models can not only successfully alleviate the vanishing gradient problem and enable deeper learning, but also lead to significantly (up to 47%) better model accuracy as compared to plain deep neural networks and traditional ML techniques for a given input materials representation in the presence of big data.

scholarly journals Laplacian networks: bounding indicator function smoothness for neural networks robustness

2021 ◽  
Vol 10 ◽  
Author(s):  
Carlos Lassance ◽  
Vincent Gripon ◽  
Antonio Ortega
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Supervised Learning ◽  
Indicator Function ◽  
Training Data ◽  
Theoretical Justification ◽  
The Past ◽  
Noisy Examples

For the past few years, deep learning (DL) robustness (i.e. the ability to maintain the same decision when inputs are subject to perturbations) has become a question of paramount importance, in particular in settings where misclassification can have dramatic consequences. To address this question, authors have proposed different approaches, such as adding regularizers or training using noisy examples. In this paper we introduce a regularizer based on the Laplacian of similarity graphs obtained from the representation of training data at each layer of the DL architecture. This regularizer penalizes large changes (across consecutive layers in the architecture) in the distance between examples of different classes, and as such enforces smooth variations of the class boundaries. We provide theoretical justification for this regularizer and demonstrate its effectiveness to improve robustness on classical supervised learning vision datasets for various types of perturbations. We also show it can be combined with existing methods to increase overall robustness.

Abstract P437: Deep Learning-based Models For Complete Atrioventricular Block Heart Rhythm Analysis

2021 ◽  
Vol 129 (Suppl_1) ◽  
Author(s):  
Dahim Choi ◽  
Nam Kyun Kim ◽  
Young H Son ◽  
Yuming Gao ◽  
Christina Sheng ◽  
...  
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Atrioventricular Block ◽  
Prediction Models ◽  
Pacemaker Implantation ◽  
Heart Rhythm ◽  
High Sensitivity ◽  
Ventricular Myocardium ◽  
True Positive Rate ◽  
Starting Point

Atrioventricular block (AVB), caused by impairment in the heart conduction system, presents extreme diversity and is associated with other complications. Only half of AVB patients require a permanent pacemaker, and the process determining the pacemaker implantation is associated with an increase in cost and patient morbidity and mortality. Thus, there is a need for models capable of accurately identifying transient or reversible causes for conduction disturbances and predicting the patient risks and the necessity of a pacemaker. Deep learning (DL) is brought to the forefront due to its prediction accuracy, and the DL-based electrocardiogram (ECG) analysis can be a breakthrough to analyze a massive amount of data. However, the current DL models are unsuitable for AVB-ECG, where the P waves are decoupled from the QRS/T waves, and a black-box nature of the DL-based model lowers the credibility of prediction models to physicians. Here, we present a real-time-capable DL-based algorithm that can identify AVB-ECG waves and automate AVB phenotyping for arrhythmogenic risk assessment. Our algorithm can analyze unformatted ECG records with abnormal patterns by integrating the two representative DL algorithms: convolutional neural networks (CNN) and recurrent neural networks (RNN). This hybrid CNN/RNN network can memorize local patterns, spatial hierarchies, and long-range temporal dependencies of ECG signals. Furthermore, by integrating parameters derived from dimension reduction analysis and heart rate variability into the hybrid layers, the algorithm can capture the P/QRS/T-specific morphological and temporal features in ECG waveforms. We evaluated the algorithm using the six AVB porcine models, where TBX18, a pacemaker transcription factor, was transduced into the ventricular myocardium to form a biological pacemaker, and an additional electronic pacemaker was transplanted as a backup pacemaker. We achieved high sensitivity (95% true positive rate) and quantified the potential risks of various pathological ECG patterns. This study may be a starting point in conducting both retrospective and prospective patient studies and will help physicians understand its decision-making workflow and find the incorrect recommendations for AVB patients.

Understanding Memories of the Past in the Context of Different Complex Neural Network Architectures

2022 ◽  
pp. 1-27
Author(s):  
Clifford Bohm ◽  
Douglas Kirkpatrick ◽  
Arend Hintze
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Mental Representations ◽  
Network Architectures ◽  
Information Theoretic ◽  
Time Points ◽  
The Past ◽  
Black Boxes ◽  
Past Experiences ◽  
Computational Systems

Abstract Deep learning (primarily using backpropagation) and neuroevolution are the preeminent methods of optimizing artificial neural networks. However, they often create black boxes that are as hard to understand as the natural brains they seek to mimic. Previous work has identified an information-theoretic tool, referred to as R, which allows us to quantify and identify mental representations in artificial cognitive systems. The use of such measures has allowed us to make previous black boxes more transparent. Here we extend R to not only identify where complex computational systems store memory about their environment but also to differentiate between different time points in the past. We show how this extended measure can identify the location of memory related to past experiences in neural networks optimized by deep learning as well as a genetic algorithm.

scholarly journals Learners Demographics Classification on MOOCs During the COVID-19: Author Profiling via Deep Learning Based on Semantic and Syntactic Representations

2021 ◽  
Vol 6 ◽  
Author(s):  
Tahani Aljohani ◽  
Alexandra I. Cristea
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Prediction Models ◽  
Short Term Memory ◽  
Methodological Approach ◽  
High Accuracy ◽  
Directional Model ◽  
Textual Representations ◽  
The One

Massive Open Online Courses (MOOCs) have become universal learning resources, and the COVID-19 pandemic is rendering these platforms even more necessary. In this paper, we seek to improve Learner Profiling (LP), i.e. estimating the demographic characteristics of learners in MOOC platforms. We have focused on examining models which show promise elsewhere, but were never examined in the LP area (deep learning models) based on effective textual representations. As LP characteristics, we predict here the employment status of learners. We compare sequential and parallel ensemble deep learning architectures based on Convolutional Neural Networks and Recurrent Neural Networks, obtaining an average high accuracy of 96.3% for our best method. Next, we predict the gender of learners based on syntactic knowledge from the text. We compare different tree-structured Long-Short-Term Memory models (as state-of-the-art candidates) and provide our novel version of a Bi-directional composition function for existing architectures. In addition, we evaluate 18 different combinations of word-level encoding and sentence-level encoding functions. Based on these results, we show that our Bi-directional model outperforms all other models and the highest accuracy result among our models is the one based on the combination of FeedForward Neural Network and the Stack-augmented Parser-Interpreter Neural Network (82.60% prediction accuracy). We argue that our prediction models recommended for both demographics characteristics examined in this study can achieve high accuracy. This is additionally also the first time a sound methodological approach toward improving accuracy for learner demographics classification on MOOCs was proposed.

scholarly journals Application of convolutional neural networks for stellar spectral classification

2019 ◽  
Vol 491 (2) ◽  
pp. 2280-2300 ◽  
Author(s):  
Kaushal Sharma ◽  
Ajit Kembhavi ◽  
Aniruddha Kembhavi ◽  
T Sivarani ◽  
Sheelu Abraham ◽  
...  
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Spectral Classification ◽  
Stellar Spectra ◽  
The Past ◽  
Training Samples ◽  
Optical Region ◽  
Spectral Libraries ◽  
Improved Accuracy

ABSTRACT Due to the ever-expanding volume of observed spectroscopic data from surveys such as SDSS and LAMOST, it has become important to apply artificial intelligence (AI) techniques for analysing stellar spectra to solve spectral classification and regression problems like the determination of stellar atmospheric parameters Teff, $\rm {\log g}$, and [Fe/H]. We propose an automated approach for the classification of stellar spectra in the optical region using convolutional neural networks (CNNs). Traditional machine learning (ML) methods with ‘shallow’ architecture (usually up to two hidden layers) have been trained for these purposes in the past. However, deep learning methods with a larger number of hidden layers allow the use of finer details in the spectrum which results in improved accuracy and better generalization. Studying finer spectral signatures also enables us to determine accurate differential stellar parameters and find rare objects. We examine various machine and deep learning algorithms like artificial neural networks, Random Forest, and CNN to classify stellar spectra using the Jacoby Atlas, ELODIE, and MILES spectral libraries as training samples. We test the performance of the trained networks on the Indo-U.S. Library of Coudé Feed Stellar Spectra (CFLIB). We show that using CNNs, we are able to lower the error up to 1.23 spectral subclasses as compared to that of two subclasses achieved in the past studies with ML approach. We further apply the trained model to classify stellar spectra retrieved from the SDSS data base with SNR > 20.

scholarly journals A Comparison of Optimization Algorithms for Deep Learning

2020 ◽  
Vol 34 (13) ◽  
pp. 2052013 ◽  
Author(s):  
Derya Soydaner
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Deep Neural Networks ◽  
Optimization Algorithms ◽  
Gradient Methods ◽  
The Past ◽  
Basic Optimization ◽  
In The Wild ◽  
Image Datasets

In recent years, we have witnessed the rise of deep learning. Deep neural networks have proved their success in many areas. However, the optimization of these networks has become more difficult as neural networks going deeper and datasets becoming bigger. Therefore, more advanced optimization algorithms have been proposed over the past years. In this study, widely used optimization algorithms for deep learning are examined in detail. To this end, these algorithms called adaptive gradient methods are implemented for both supervised and unsupervised tasks. The behavior of the algorithms during training and results on four image datasets, namely, MNIST, CIFAR-10, Kaggle Flowers and Labeled Faces in the Wild are compared by pointing out their differences against basic optimization algorithms.

Sign in / Sign up

Export Citation Format

Share Document