Peel Learning for Pathway-Related Outcome Prediction

Abstract Motivation Traditional regression models are limited in outcome prediction due to their parametric nature. Current deep learning methods allow for various effects and interactions and have shown improved performance, but they typically need to be trained on a large amount of data to obtain reliable results. Gene expression studies often have small sample sizes but high dimensional correlated predictors so that traditional deep learning methods are not readily applicable. Results In this paper, we proposed peel learning, a novel neural network that incorporates the prior relationship among genes. In each layer of learning, overall structure is peeled into multiple local substructures. Within the substructure, dependency among variables is reduced through linear projections. The overall structure is gradually simplified over layers and weight parameters are optimized through a revised backpropagation. We applied PL to a small lung transplantation study to predict recipients’ post-surgery primary graft dysfunction using donors’ gene expressions within several immunology pathways, where PL showed improved prediction accuracy compared to conventional penalized regression, classification trees, feed-forward neural network, and a neural network assuming prior network structure. Through simulation studies, we also demonstrated the advantage of adding specific structure among predictor variables in neural network, over no or uniform group structure, which is more favorable in smaller studies. The empirical evidence is consistent with our theoretical proof of improved upper bound of PL’s complexity over ordinary neural networks. Availability and Implementation PL algorithm was implemented in Python and the open-source code and instruction will be available at https://github.com/Likelyt/Peel-Learning

Download Full-text

A New Generative Neural Network for Bearing Fault Diagnosis with Imbalanced Data

Shock and Vibration ◽

10.1155/2020/8869648 ◽

2020 ◽

Vol 2020 ◽

pp. 1-17

Author(s):

Wei You ◽

Changqing Shen ◽

Liang Chen ◽

Hongbo Que ◽

Weiguo Huang ◽

...

Keyword(s):

Neural Network ◽

Deep Learning ◽

Fault Diagnosis ◽

Imbalanced Data ◽

Small Sample ◽

Convolution Neural Network ◽

Learning Methods ◽

Research Attention ◽

Bearing Fault ◽

Bearing Fault Diagnosis

Intelligent bearing fault diagnosis has received much research attention in the field of rotary machinery systems where miscellaneous deep learning methods are generally applied. Among these methods, convolution neural network is particularly powerful because of its ability to learn fruitful features from the original data. However, normal convolutions cannot fully utilize the information along the data flow while the features are being abstracted in deeper layers. To address this problem, a new supervised learning model is proposed for small sample size bearing fault diagnosis with consideration of imbalanced data. This model, which is developed based on a convolution neural network, has a high generalization ability, and its performance is verified by conducting two experiments that use data collected from a self-made bearing test rig. The proposed model demonstrates a favorable performance and is more effective and robust than other deep learning methods.

Download Full-text

A Review of Plant Phenotypic Image Recognition Technology Based on Deep Learning

Electronics ◽

10.3390/electronics10010081 ◽

2021 ◽

Vol 10 (1) ◽

pp. 81

Author(s):

Jianbin Xiong ◽

Dezheng Yu ◽

Shuangyin Liu ◽

Lei Shu ◽

Xiaochan Wang ◽

...

Keyword(s):

Neural Network ◽

Deep Learning ◽

Convolutional Neural Network ◽

Plant Species ◽

Image Recognition ◽

Recurrent Neural Network ◽

Plant Diseases ◽

Learning Methods ◽

Smart Agriculture ◽

Important Branch

Plant phenotypic image recognition (PPIR) is an important branch of smart agriculture. In recent years, deep learning has achieved significant breakthroughs in image recognition. Consequently, PPIR technology that is based on deep learning is becoming increasingly popular. First, this paper introduces the development and application of PPIR technology, followed by its classification and analysis. Second, it presents the theory of four types of deep learning methods and their applications in PPIR. These methods include the convolutional neural network, deep belief network, recurrent neural network, and stacked autoencoder, and they are applied to identify plant species, diagnose plant diseases, etc. Finally, the difficulties and challenges of deep learning in PPIR are discussed.

Download Full-text

Detecting the Absence of Lung Sliding in Lung Ultrasounds Using Deep Learning

Applied Sciences ◽

10.3390/app11156976 ◽

2021 ◽

Vol 11 (15) ◽

pp. 6976

Author(s):

Miroslav Jaščur ◽

Marek Bundzel ◽

Marek Malík ◽

Anton Dzian ◽

Norbert Ferenčík ◽

...

Keyword(s):

Neural Network ◽

Deep Learning ◽

Lung Ultrasound ◽

Video Recording ◽

X Rays ◽

Surgery Complications ◽

Temporal Dimension ◽

Sliding Motion ◽

Post Surgery ◽

Lung Sliding

Certain post-thoracic surgery complications are monitored in a standard manner using methods that employ ionising radiation. A need to automatise the diagnostic procedure has now arisen following the clinical trial of a novel lung ultrasound examination procedure that can replace X-rays. Deep learning was used as a powerful tool for lung ultrasound analysis. We present a novel deep-learning method, automated M-mode classification, to detect the absence of lung sliding motion in lung ultrasound. Automated M-mode classification leverages semantic segmentation to select 2D slices across the temporal dimension of the video recording. These 2D slices are the input for a convolutional neural network, and the output of the neural network indicates the presence or absence of lung sliding in the given time slot. We aggregate the partial predictions over the entire video recording to determine whether the subject has developed post-surgery complications. With a 64-frame version of this architecture, we detected lung sliding on average with a balanced accuracy of 89%, sensitivity of 82%, and specificity of 92%. Automated M-mode classification is suitable for lung sliding detection from clinical lung ultrasound videos. Furthermore, in lung ultrasound videos, we recommend using time windows between 0.53 and 2.13 s for the classification of lung sliding motion followed by aggregation.

Download Full-text

Deep Learning Methods for Heart Sounds Classification: A Systematic Review

Entropy ◽

10.3390/e23060667 ◽

2021 ◽

Vol 23 (6) ◽

pp. 667

Author(s):

Wei Chen ◽

Qiang Sun ◽

Xiaomin Chen ◽

Gangcai Xie ◽

Huiqun Wu ◽

...

Keyword(s):

Neural Network ◽

Systematic Review ◽

Deep Learning ◽

Heart Sounds ◽

Learning Approaches ◽

Automated Classification ◽

Learning Methods ◽

Artificial Intelligence Technology ◽

Medical Big Data ◽

Effective Models

The automated classification of heart sounds plays a significant role in the diagnosis of cardiovascular diseases (CVDs). With the recent introduction of medical big data and artificial intelligence technology, there has been an increased focus on the development of deep learning approaches for heart sound classification. However, despite significant achievements in this field, there are still limitations due to insufficient data, inefficient training, and the unavailability of effective models. With the aim of improving the accuracy of heart sounds classification, an in-depth systematic review and an analysis of existing deep learning methods were performed in the present study, with an emphasis on the convolutional neural network (CNN) and recurrent neural network (RNN) methods developed over the last five years. This paper also discusses the challenges and expected future trends in the application of deep learning to heart sounds classification with the objective of providing an essential reference for further study.

Download Full-text

Validating Deep Neural Networks for Online Decoding of Motor Imagery Movements from EEG Signals

Sensors ◽

10.3390/s19010210 ◽

2019 ◽

Vol 19 (1) ◽

pp. 210 ◽

Cited By ~ 32

Author(s):

Zied Tayeb ◽

Juri Fedjaev ◽

Nejla Ghaboosi ◽

Christoph Richter ◽

Lukas Everding ◽

...

Keyword(s):

Neural Network ◽

Machine Learning ◽

Deep Learning ◽

Convolutional Neural Network ◽

Motor Imagery ◽

Classification Performance ◽

Feature Engineering ◽

Learning Models ◽

Eeg Signals ◽

Learning Methods

Non-invasive, electroencephalography (EEG)-based brain-computer interfaces (BCIs) on motor imagery movements translate the subject’s motor intention into control signals through classifying the EEG patterns caused by different imagination tasks, e.g., hand movements. This type of BCI has been widely studied and used as an alternative mode of communication and environmental control for disabled patients, such as those suffering from a brainstem stroke or a spinal cord injury (SCI). Notwithstanding the success of traditional machine learning methods in classifying EEG signals, these methods still rely on hand-crafted features. The extraction of such features is a difficult task due to the high non-stationarity of EEG signals, which is a major cause by the stagnating progress in classification performance. Remarkable advances in deep learning methods allow end-to-end learning without any feature engineering, which could benefit BCI motor imagery applications. We developed three deep learning models: (1) A long short-term memory (LSTM); (2) a spectrogram-based convolutional neural network model (CNN); and (3) a recurrent convolutional neural network (RCNN), for decoding motor imagery movements directly from raw EEG signals without (any manual) feature engineering. Results were evaluated on our own publicly available, EEG data collected from 20 subjects and on an existing dataset known as 2b EEG dataset from “BCI Competition IV”. Overall, better classification performance was achieved with deep learning models compared to state-of-the art machine learning techniques, which could chart a route ahead for developing new robust techniques for EEG signal decoding. We underpin this point by demonstrating the successful real-time control of a robotic arm using our CNN based BCI.

Download Full-text

Optimizing Deep Learning Methods in Neural Network Architectures

International Review of Automatic Control (IREACO) ◽

10.15866/ireaco.v14i2.20591 ◽

2021 ◽

Vol 14 (2) ◽

pp. 93

Author(s):

Kristina Gorshkova ◽

Victoria Zueva ◽

Maria Kuznetsova ◽

Larisa Tugashova

Keyword(s):

Neural Network ◽

Deep Learning ◽

Network Architectures ◽

Learning Methods ◽

Neural Network Architectures

Download Full-text

Deep Learning for Caries Detection and Classification

Diagnostics ◽

10.3390/diagnostics11091672 ◽

2021 ◽

Vol 11 (9) ◽

pp. 1672

Author(s):

Luya Lian ◽

Tianer Zhu ◽

Fudong Zhu ◽

Haihua Zhu

Keyword(s):

Neural Network ◽

Deep Learning ◽

Disease Diagnosis ◽

Validation Dataset ◽

Reference Dataset ◽

Dice Coefficient ◽

Learning Methods ◽

Test Dataset ◽

The Impact ◽

Caries Lesions

Objectives: Deep learning methods have achieved impressive diagnostic performance in the field of radiology. The current study aimed to use deep learning methods to detect caries lesions, classify different radiographic extensions on panoramic films, and compare the classification results with those of expert dentists. Methods: A total of 1160 dental panoramic films were evaluated by three expert dentists. All caries lesions in the films were marked with circles, whose combination was defined as the reference dataset. A training and validation dataset (1071) and a test dataset (89) were then established from the reference dataset. A convolutional neural network, called nnU-Net, was applied to detect caries lesions, and DenseNet121 was applied to classify the lesions according to their depths (dentin lesions in the outer, middle, or inner third D1/2/3 of dentin). The performance of the test dataset in the trained nnU-Net and DenseNet121 models was compared with the results of six expert dentists in terms of the intersection over union (IoU), Dice coefficient, accuracy, precision, recall, negative predictive value (NPV), and F1-score metrics. Results: nnU-Net yielded caries lesion segmentation IoU and Dice coefficient values of 0.785 and 0.663, respectively, and the accuracy and recall rate of nnU-Net were 0.986 and 0.821, respectively. The results of the expert dentists and the neural network were shown to be no different in terms of accuracy, precision, recall, NPV, and F1-score. For caries depth classification, DenseNet121 showed an overall accuracy of 0.957 for D1 lesions, 0.832 for D2 lesions, and 0.863 for D3 lesions. The recall results of the D1/D2/D3 lesions were 0.765, 0.652, and 0.918, respectively. All metric values, including accuracy, precision, recall, NPV, and F1-score values, were proven to be no different from those of the experienced dentists. Conclusion: In detecting and classifying caries lesions on dental panoramic radiographs, the performance of deep learning methods was similar to that of expert dentists. The impact of applying these well-trained neural networks for disease diagnosis and treatment decision making should be explored.

Download Full-text

A Comparative Study of Bitcoin Price Prediction Using Deep Learning

Mathematics ◽

10.3390/math7100898 ◽

2019 ◽

Vol 7 (10) ◽

pp. 898 ◽

Cited By ~ 12

Author(s):

Suhwan Ji ◽

Jongmin Kim ◽

Hyeonseung Im

Keyword(s):

Neural Network ◽

Deep Learning ◽

Prediction Models ◽

Short Term Memory ◽

Learning Methods ◽

The Public ◽

Price Prediction ◽

Profitability Analysis ◽

The Media ◽

Long Short Term Memory

Bitcoin has recently received a lot of attention from the media and the public due to its recent price surge and crash. Correspondingly, many researchers have investigated various factors that affect the Bitcoin price and the patterns behind its fluctuations, in particular, using various machine learning methods. In this paper, we study and compare various state-of-the-art deep learning methods such as a deep neural network (DNN), a long short-term memory (LSTM) model, a convolutional neural network, a deep residual network, and their combinations for Bitcoin price prediction. Experimental results showed that although LSTM-based prediction models slightly outperformed the other prediction models for Bitcoin price prediction (regression), DNN-based models performed the best for price ups and downs prediction (classification). In addition, a simple profitability analysis showed that classification models were more effective than regression models for algorithmic trading. Overall, the performances of the proposed deep learning-based prediction models were comparable.

Download Full-text

A Novel Outdoor Positioning Technique Using LTE Network Fingerprints

Sensors ◽

10.3390/s20061691 ◽

2020 ◽

Vol 20 (6) ◽

pp. 1691 ◽

Cited By ~ 1

Author(s):

Da Li ◽

Yingke Lei ◽

Haichuan Zhang

Keyword(s):

Neural Network ◽

Deep Learning ◽

Transfer Learning ◽

Feed Forward Neural Network ◽

Research Attention ◽

Outdoor Environments ◽

Dispersion Problem ◽

Fingerprint Database ◽

The Internet Of Things

In recent years, wireless-based fingerprint positioning has attracted increasing research attention owing to its position-related features and applications in the Internet of Things (IoT). In this paper, by leveraging long-term evolution (LTE) signals, a novel deep-learning-based fingerprint positioning approach is proposed to solve the problem of outdoor positioning. Considering the outstanding performance of deep learning in image classification, LTE signal measurements are converted into location grayscale images to form a fingerprint database. In order to deal with the instability of LTE signals, prevent the gradient dispersion problem, and increase the robustness of the proposed deep neural network (DNN), the following methods are adopted: First, cross-entropy is used as the loss function of the DNN. Second, the learning rate of the proposed DNN is dynamically adjusted. Third, this paper adopted several data enhancement techniques. To find the best positioning fingerprint and method, three types of fingerprint and five positioning models are compared. Finally, by using a deep residual network (Resnet) and transfer learning, a hierarchical structure training method is proposed. The proposed Resnet is used to train with the united fingerprint image database to obtain a positioning model called a coarse localizer. By using the prior knowledge of the pretrained Resnet, feed-forward neural network (FFNN)-based transfer learning is used to train with the united fingerprint database to obtain a better positioning model, called a fine localizer. The experimental results convincingly show that the proposed DNN can automatically learn the location features of LTE signals and achieve satisfactory positioning accuracy in outdoor environments.

Download Full-text

A Statistics and Deep Learning Hybrid Method for Multivariate Time Series Forecasting and Mortality Modeling

Forecasting ◽

10.3390/forecast4010001 ◽

2021 ◽

Vol 4 (1) ◽

pp. 1-25

Author(s):

Thabang Mathonsi ◽

Terence L. van Zyl

Keyword(s):

Neural Network ◽

Deep Learning ◽

Recurrent Neural Network ◽

Short Term Memory ◽

Hybrid Methods ◽

Prediction Interval ◽

Hybrid Approach ◽

Computational Cost ◽

Exponential Smoothing ◽

Learning Methods

Hybrid methods have been shown to outperform pure statistical and pure deep learning methods at forecasting tasks and quantifying the associated uncertainty with those forecasts (prediction intervals). One example is Exponential Smoothing Recurrent Neural Network (ES-RNN), a hybrid between a statistical forecasting model and a recurrent neural network variant. ES-RNN achieves a 9.4% improvement in absolute error in the Makridakis-4 Forecasting Competition. This improvement and similar outperformance from other hybrid models have primarily been demonstrated only on univariate datasets. Difficulties with applying hybrid forecast methods to multivariate data include (i) the high computational cost involved in hyperparameter tuning for models that are not parsimonious, (ii) challenges associated with auto-correlation inherent in the data, as well as (iii) complex dependency (cross-correlation) between the covariates that may be hard to capture. This paper presents Multivariate Exponential Smoothing Long Short Term Memory (MES-LSTM), a generalized multivariate extension to ES-RNN, that overcomes these challenges. MES-LSTM utilizes a vectorized implementation. We test MES-LSTM on several aggregated coronavirus disease of 2019 (COVID-19) morbidity datasets and find our hybrid approach shows consistent, significant improvement over pure statistical and deep learning methods at forecast accuracy and prediction interval construction.

Download Full-text