scholarly journals Modelling cross-sectional tabular data using convolutional neural networks: Prediction of corporate bankruptcy in Poland

2021 ◽  
Vol 8 (55) ◽  
pp. 352-377
Author(s):  
Aneta Dzik-Walczak ◽  
Maciej Odziemczyk
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Random Search ◽  
Learning Curves ◽  
Corporate Bankruptcy ◽  
Tabular Data ◽  
Cross Sectional ◽  
Data Transformations ◽  
Convolutional Networks ◽  
Observation Vector

Abstract The paper deals with the topic of modelling the probability of bankruptcy of Polish enterprises using convolutional neural networks. Convolutional networks take images as input, so it was thus necessary to apply the method of converting the observation vector to a matrix. Benchmarks for convolutional networks were logit models, random forests, XGBoost, and dense neural networks. Hyperparameters and model architecture were selected based on a random search and analysis of learning curves and experiments in folded, stratified cross-validation. In addition, the sensitivity of the results to data preprocessing was investigated. It was found that convolutional neural networks can be used to analyze cross-sectional tabular data, especially for the problem of modelling the probability of corporate bankruptcy. In order to achieve good results with models based on parameters updated by a gradient (neural networks and logit), it is necessary to use appropriate preprocessing techniques. Models based on decision trees have been shown to be insensitive to the data transformations used.

scholarly journals Publisher Correction: Converting tabular data into images for deep learning with convolutional neural networks

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Yitan Zhu ◽  
Thomas Brettin ◽  
Fangfang Xia ◽  
Alexander Partin ◽  
Maulik Shukla ◽  
...  
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Tabular Data

scholarly journals Phase harmonic correlations and convolutional neural networks

2019 ◽  
Vol 9 (3) ◽  
pp. 721-747 ◽  
Author(s):  
Stéphane Mallat ◽  
Sixin Zhang ◽  
Gaspar Rochette
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Coherent Structures ◽  
Phase Dependence ◽  
Wavelet Filters ◽  
Convolutional Networks ◽  
Nonlinear Phase ◽  
Phase Filter ◽  
Multiplicative Operator ◽  
Space Frequency

Abstract A major issue in harmonic analysis is to capture the phase dependence of frequency representations, which carries important signal properties. It seems that convolutional neural networks have found a way. Over time-series and images, convolutional networks often learn a first layer of filters that are well localized in the frequency domain, with different phases. We show that a rectifier then acts as a filter on the phase of the resulting coefficients. It computes signal descriptors that are local in space, frequency and phase. The nonlinear phase filter becomes a multiplicative operator over phase harmonics computed with a Fourier transform along the phase. We prove that it defines a bi-Lipschitz and invertible representation. The correlations of phase harmonics coefficients characterize coherent structures from their phase dependence across frequencies. For wavelet filters, we show numerically that signals having sparse wavelet coefficients can be recovered from few phase harmonic correlations, which provide a compressive representation.

scholarly journals Single shot detector application for image disease localization

2021 ◽  
Author(s):  
shrikant pawar ◽  
Aditya Stanam ◽  
Rushikesh Chopade
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Feature Detection ◽  
Single Shot ◽  
X Ray ◽  
Convolutional Networks ◽  
Fully Convolutional Networks ◽  
Bounding Box ◽  
Unique Approach ◽  
Detection And Localization

Bounding box algorithms are useful in localization of image patterns. Recently, utilization of convolutional neural networks on X-ray images has proven a promising disease prediction technique. However, pattern localization over prediction has always been a challenging task with inconsistent coordinates, sizes, resolution and capture positions of an image. Several model architectures like Fast R-CNN, Faster R-CNN, Histogram of Oriented Gradients (HOG), You only look once (YOLO), Region-based Convolutional Neural Networks (R-CNN), Region-based Fully Convolutional Networks (R-FCN), Single Shot Detector (SSD), etc. are used for object detection and localization in modern-day computer vision applications. SSD and region-based detectors like Fast R-CNN or Faster R-CNN are very similar in design and implementation, but SSD have shown to work efficiently with larger frames per second (FPS) and lower resolution images. In this article, we present a unique approach of SSD with a VGG-16 network as a backbone for feature detection of bounding box algorithm to predict the location of an anomaly within chest X-ray image.

scholarly journals AN APPROACH TO ACCELERATE THE TRAINING OF CONVOLUTIONAL NEURAL NETWORKS BY TUNING THE HYPERPARAMETERS OF LEARNING

2020 ◽  
Vol 2 (2) ◽  
pp. 32-37
Author(s):  
P. RADIUK ◽  
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Video Processing ◽  
Network Architecture ◽  
Random Search ◽  
Machine Learning Algorithms ◽  
Language Recognition ◽  
Data Set ◽  
Training Time

Over the last decade, a set of machine learning algorithms called deep learning has led to significant improvements in computer vision, natural language recognition and processing. This has led to the widespread use of a variety of commercial, learning-based products in various fields of human activity. Despite this success, the use of deep neural networks remains a black box. Today, the process of setting hyperparameters and designing a network architecture requires experience and a lot of trial and error and is based more on chance than on a scientific approach. At the same time, the task of simplifying deep learning is extremely urgent. To date, no simple ways have been invented to establish the optimal values of learning hyperparameters, namely learning speed, sample size, data set, learning pulse, and weight loss. Grid search and random search of hyperparameter space are extremely resource intensive. The choice of hyperparameters is critical for the training time and the final result. In addition, experts often choose one of the standard architectures (for example, ResNets and ready-made sets of hyperparameters. However, such kits are usually suboptimal for specific practical tasks. The presented work offers an approach to finding the optimal set of hyperparameters of learning ZNM. An integrated approach to all hyperparameters is valuable because there is an interdependence between them. The aim of the work is to develop an approach for setting a set of hyperparameters, which will reduce the time spent during the design of ZNM and ensure the efficiency of its work. In recent decades, the introduction of deep learning methods, in particular convolutional neural networks (CNNs), has led to impressive success in image and video processing. However, the training of CNN has been commonly mostly based on the employment of quasi-optimal hyperparameters. Such an approach usually requires huge computational and time costs to train the network and does not guarantee a satisfactory result. However, hyperparameters play a crucial role in the effectiveness of CNN, as diverse hyperparameters lead to models with significantly different characteristics. Poorly selected hyperparameters generally lead to low model performance. The issue of choosing optimal hyperparameters for CNN has not been resolved yet. The presented work proposes several practical approaches to setting hyperparameters, which allows reducing training time and increasing the accuracy of the model. The article considers the function of training validation loss during underfitting and overfitting. There are guidelines in the end to reach the optimization point. The paper also considers the regulation of learning rate and momentum to accelerate network training. All experiments are based on the widespread CIFAR-10 and CIFAR-100 datasets.

scholarly journals Automated Detection of Interictal Epileptiform Discharges from Scalp Electroencephalograms by Convolutional Neural Networks

2020 ◽  
Vol 30 (11) ◽  
pp. 2050030 ◽  
Author(s):  
John Thomas ◽  
Jing Jin ◽  
Prasanth Thangavel ◽  
Elham Bagheri ◽  
Rajamanickam Yuvaraj ◽  
...  
Keyword(s):  
Neural Networks ◽  
South Carolina ◽  
Convolutional Neural Networks ◽  
Massachusetts General Hospital ◽  
Learning Curves ◽  
University Hospital ◽  
Superior Performance ◽  
Epileptiform Discharges ◽  
Interictal Epileptiform Discharges ◽  
Eeg Recordings

Visual evaluation of electroencephalogram (EEG) for Interictal Epileptiform Discharges (IEDs) as distinctive biomarkers of epilepsy has various limitations, including time-consuming reviews, steep learning curves, interobserver variability, and the need for specialized experts. The development of an automated IED detector is necessary to provide a faster and reliable diagnosis of epilepsy. In this paper, we propose an automated IED detector based on Convolutional Neural Networks (CNNs). We have evaluated the proposed IED detector on a sizable database of 554 scalp EEG recordings (84 epileptic patients and 461 nonepileptic subjects) recorded at Massachusetts General Hospital (MGH), Boston. The proposed CNN IED detector has achieved superior performance in comparison with conventional methods with a mean cross-validation area under the precision–recall curve (AUPRC) of 0.838[Formula: see text]±[Formula: see text]0.040 and false detection rate of 0.2[Formula: see text]±[Formula: see text]0.11 per minute for a sensitivity of 80%. We demonstrated the proposed system to be noninferior to 30 neurologists on a dataset from the Medical University of South Carolina (MUSC). Further, we clinically validated the system at National University Hospital (NUH), Singapore, with an agreement accuracy of 81.41% with a clinical expert. Moreover, the proposed system can be applied to EEG recordings with any arbitrary number of channels.

scholarly journals Reinforcement Learning for Hyperparameter Tuning in Deep Learning-based Side-channel Analysis

2021 ◽  
pp. 677-707
Author(s):  
Jorai Rijsdijk ◽  
Lichao Wu ◽  
Guilherme Perin ◽  
Stjepan Picek
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Reinforcement Learning ◽  
Convolutional Neural Networks ◽  
Random Search ◽  
High Price ◽  
Side Channel ◽  
Q Learning ◽  
Reward Functions

Deep learning represents a powerful set of techniques for profiling sidechannel analysis. The results in the last few years show that neural network architectures like multilayer perceptron and convolutional neural networks give strong attack performance where it is possible to break targets protected with various countermeasures. Considering that deep learning techniques commonly have a plethora of hyperparameters to tune, it is clear that such top attack results can come with a high price in preparing the attack. This is especially problematic as the side-channel community commonly uses random search or grid search techniques to look for the best hyperparameters.In this paper, we propose to use reinforcement learning to tune the convolutional neural network hyperparameters. In our framework, we investigate the Q-Learning paradigm and develop two reward functions that use side-channel metrics. We mount an investigation on three commonly used datasets and two leakage models where the results show that reinforcement learning can find convolutional neural networks exhibiting top performance while having small numbers of trainable parameters. We note that our approach is automated and can be easily adapted to different datasets. Several of our newly developed architectures outperform the current state-of-the-art results. Finally, we make our source code publicly available. https://github.com/AISyLab/Reinforcement-Learning-for-SCA

scholarly journals Rectified Binary Convolutional Networks for Enhancing the Performance of 1-bit DCNNs

2019 ◽  
Author(s):  
Chunlei Liu ◽  
Wenrui Ding ◽  
Xin Xia ◽  
Yuan Hu ◽  
Baochang Zhang ◽  
...  
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Superior Performance ◽  
Feature Maps ◽  
Unified Framework ◽  
Deep Convolutional Neural Networks ◽  
Convolutional Networks ◽  
Computation Efficiency ◽  
Significant Performance ◽  
Binary Network

Binarized  convolutional neural networks (BCNNs) are widely used to improve memory and computation efficiency of deep convolutional neural networks (DCNNs) for mobile and AI chips based applications. However, current BCNNs are not able to fully explore their corresponding full-precision models, causing a significant performance gap between them. In this paper, we propose rectified binary convolutional networks (RBCNs), towards optimized BCNNs, by combining full-precision kernels and feature maps to rectify the binarization process in a unified framework. In particular, we use a GAN to train the 1-bit binary network with the guidance of its corresponding full-precision model, which significantly improves the performance of BCNNs. The rectified convolutional layers are generic and flexible, and can be easily incorporated into existing DCNNs such as WideResNets and ResNets. Extensive experiments demonstrate the superior performance of the proposed RBCNs over state-of-the-art BCNNs. In particular, our method shows strong generalization on the object tracking task.

scholarly journals Alzheimer’s Detection through 3D Convolutional Neural Networks

2021 ◽  
Vol 34 (1) ◽  
Author(s):  
Ryan Hogan ◽  
Christoforos Christoforou
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Three Dimensional ◽  
Multilayer Perceptrons ◽  
Brain Scans ◽  
Convolutional Networks ◽  
3D Network ◽  
Efficient Resource ◽  
Mri Scans

To inform a proper diagnosis and understanding of Alzheimer’s Disease (AD), deep learning has emerged as an alternate approach for detecting physical brain changes within magnetic resonance imaging (MRI). The advancement of deep learning within biomedical imaging, particularly in MRI scans, has proven to be an efficient resource for abnormality detection while utilizing convolutional neural networks (CNN) to perform feature mapping within multilayer perceptrons. In this study, we aim to test the feasibility of using three-dimensional convolutional neural networks to identify neurophysiological degeneration in the entire-brain scans that differentiate between AD patients and controls. In particular, we propose and train a 3D-CNN model to classify between MRI scans of cognitively-healthy individuals and AD patients. We validate our proposed model on a large dataset composed of more than seven hundred MRI scans (half AD). Our results show a validation accuracy of 79% which is at par with the current state-of-the-art. The benefits of our proposed 3D network are that it can assist in the exploration and detection of AD by mapping the complex heterogeneity of the brain, particularly in the limbic system and temporal lobe. The goal of this research is to measure the efficacy and predictability of 3D convolutional networks in detecting the progression of neurodegeneration within MRI brain scans of HC and AD patients.

Sign in / Sign up

Export Citation Format

Share Document