scholarly journals A Model Visualization-based Approach for Insight into Waveforms and Spectra Learned by CNNs

2021 ◽  
Author(s):  
Charles A Ellis ◽  
Robyn L Miller ◽  
Vince D Calhoun
Keyword(s):  
Neural Networks ◽  
Time Series Data ◽  
Sleep Stage ◽  
Series Data ◽  
Spectral Features ◽  
Classifier Performance ◽  
Sleep Stage Classification ◽  
Significant Step ◽  
Model Visualization ◽  
Insight Into

Recent years have shown a growth in the application of deep learning architectures such as convolutional neural networks (CNNs), to electrophysiology analysis. However, using neural networks with raw time-series data makes explainability a significant challenge. Multiple explainability approaches have been developed for insight into the spectral features learned by CNNs from EEG. However, across electrophysiology modalities, and even within EEG, there are many unique waveforms of clinical relevance. Existing methods that provide insight into waveforms learned by CNNs are of questionable utility. In this study, we present a novel model visualization-based approach that analyzes the filters in the first convolutional layer of the network. To our knowledge, this is the first method focused on extracting explainable information from EEG waveforms learned by CNNs while also providing insight into the learned spectral features. We demonstrate the viability of our approach within the context of automated sleep stage classification, a well-characterized domain that can help validate our approach. We identify 3 subgroups of filters with distinct spectral properties, determine the relative importance of each group of filters, and identify several unique waveforms learned by the classifier that were vital to the classifier performance. Our approach represents a significant step forward in explainability for electrophysiology classifiers, which we also hope will be useful for providing insights in future studies.

scholarly journals A Novel Activation Maximization-based Approach for Insight into Electrophysiology Classifiers

2021 ◽  
Author(s):  
Charles A Ellis ◽  
Mohammad S.E. Sendi ◽  
Robyn L Miller ◽  
Vince D Calhoun
Keyword(s):  
Time Series ◽  
Deep Learning ◽  
Sleep Stage ◽  
Motion Artifact ◽  
Spectral Features ◽  
Attribute Importance ◽  
Learning Classifiers ◽  
Sleep Stage Classification ◽  
Long Time ◽  
Insight Into

Spectral analysis remains a hallmark approach for gaining insight into electrophysiology modalities like electroencephalography (EEG). As the field of deep learning has progressed, more studies have begun to train deep learning classifiers on raw EEG data, which presents unique problems for explainability. A growing number of studies have presented explainability approaches that provide insight into the spectral features learned by deep learning classifiers. However, existing approaches only attribute importance to different frequency bands. Most of the methods cannot provide insight into the actual spectral values or the relationship between spectral features that models have learned. Here, we present a novel adaptation of activation maximization for electrophysiology time-series that generates samples that indicate the features learned by classifiers by optimizing their spectral content. We evaluate our approach within the context of EEG sleep stage classification with a convolutional neural network, and we find that our approach is able to identify spectral patterns known to be associated with each sleep stage. We also find surprising results suggesting that our classifier may have prioritized the use of eye and motion artifact when identifying Awake samples. Our approach is the first adaptation of activation maximization to the domain of raw electrophysiology classification. Additionally, our approach has implications for explaining any classifier trained on highly dynamic, long time-series.

Neural Network Techniques for Time Series Prediction: A Review

2019 ◽  
Vol 3 (3) ◽  
Author(s):  
Muhammad Faheem Mushtaq ◽  
Urooj Akram ◽  
Muhammad Aamir ◽  
Haseeb Ali ◽  
Muhammad Zulqarnain
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Time Series ◽  
Time Series Data ◽  
Weather Prediction ◽  
Time Series Prediction ◽  
Series Data ◽  
Prediction Problem ◽  
Neural Network Models ◽  
Physical Time

It is important to predict a time series because many problems that are related to prediction such as health prediction problem, climate change prediction problem and weather prediction problem include a time component. To solve the time series prediction problem various techniques have been developed over many years to enhance the accuracy of forecasting. This paper presents a review of the prediction of physical time series applications using the neural network models. Neural Networks (NN) have appeared as an effective tool for forecasting of time series.  Moreover, to resolve the problems related to time series data, there is a need of network with single layer trainable weights that is Higher Order Neural Network (HONN) which can perform nonlinearity mapping of input-output. So, the developers are focusing on HONN that has been recently considered to develop the input representation spaces broadly. The HONN model has the ability of functional mapping which determined through some time series problems and it shows the more benefits as compared to conventional Artificial Neural Networks (ANN). The goal of this research is to present the reader awareness about HONN for physical time series prediction, to highlight some benefits and challenges using HONN.

Neural networks approached for modelling river suspended sediment concentration due to tropical storms

2013 ◽  
Vol 11 (4) ◽  
pp. 457-466
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Suspended Sediment ◽  
Suspended Sediment Concentration ◽  
Time Series Data ◽  
Water Discharge ◽  
Sediment Concentration ◽  
Series Data ◽  
Generalized Regression Neural Network ◽  
Event Based

Artificial neural networks are one of the advanced technologies employed in hydrology modelling. This paper investigates the potential of two algorithm networks, the feed forward backpropagation (BP) and generalized regression neural network (GRNN) in comparison with the classical regression for modelling the event-based suspended sediment concentration at Jiasian diversion weir in Southern Taiwan. For this study, the hourly time series data comprised of water discharge, turbidity and suspended sediment concentration during the storm events in the year of 2002 are taken into account in the models. The statistical performances comparison showed that both BP and GRNN are superior to the classical regression in the weir sediment modelling. Additionally, the turbidity was found to be a dominant input variable over the water discharge for suspended sediment concentration estimation. Statistically, both neural network models can be successfully applied for the event-based suspended sediment concentration modelling in the weir studied herein when few data are available.

A review of irregular time series data handling with gated recurrent neural networks

2021 ◽  
Vol 441 ◽  
pp. 161-178
Author(s):  
Philip B. Weerakody ◽  
Kok Wai Wong ◽  
Guanjin Wang ◽  
Wendell Ela
Keyword(s):  
Neural Networks ◽  
Time Series ◽  
Recurrent Neural Networks ◽  
Time Series Data ◽  
Series Data ◽  
Data Handling

scholarly journals Ship-Collision Avoidance Decision-Making Learning of Unmanned Surface Vehicles with Automatic Identification System Data Based on Encoder—Decoder Automatic-Response Neural Networks

2020 ◽  
Vol 8 (10) ◽  
pp. 754
Author(s):  
Miao Gao ◽  
Guo-You Shi
Keyword(s):  
Neural Networks ◽  
Decision Making ◽  
Time Series ◽  
Collision Avoidance ◽  
Time Series Data ◽  
Series Data ◽  
Automatic Identification ◽  
Identification System ◽  
Automatic Response ◽  
Ship Collision

Intelligent unmanned surface vehicle (USV) collision avoidance is a complex inference problem based on current navigation status. This requires simultaneous processing of the input sequences and generation of the response sequences. The automatic identification system (AIS) encounter data mainly include the time-series data of two AIS sets, which exhibit a one-to-one mapping relation. Herein, an encoder–decoder automatic-response neural network is designed and implemented based on the sequence-to-sequence (Seq2Seq) structure to simultaneously process the two AIS encounter trajectory sequences. Furthermore, this model is combined with the bidirectional long short-term memory recurrent neural networks (Bi-LSTM RNN) to obtain a network framework for processing the time-series data to obtain ship-collision avoidance decisions based on big data. The encoder–decoder neural networks were trained based on the AIS data obtained in 2018 from Zhoushan Port to achieve ship collision avoidance decision-making learning. The results indicated that the encoder–decoder neural networks can be used to effectively formulate the sequence of the collision avoidance decision of the USV. Thus, this study significantly contributes to the increased efficiency and safety of maritime transportation. The proposed method can potentially be applied to the USV technology and intelligent collision-avoidance systems.

Time Series Data Analysis by Ultra-High Frequency Trigonometric Higher Order Neural Networks

2021 ◽  
pp. 218-261
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Exchange Rates ◽  
High Frequency ◽  
Time Series Data ◽  
Higher Order ◽  
Series Data ◽  
Ultra High Frequency ◽  
Higher Order Neural Networks ◽  
Time Series Data Analysis

This chapter develops a new nonlinear model, ultra high frequency trigonometric higher order neural networks (UTHONN) for time series data analysis. UTHONN includes three models: UCSHONN (ultra high frequency sine and cosine higher order neural networks) models, UCCHONN (ultra high frequency cosine and cosine higher order neural networks) models, and USSHONN (ultra high frequency sine and sine higher order neural networks) models. Results show that UTHONN models are 3 to 12% better than equilibrium real exchange rates (ERER) model, and 4–9% better than other polynomial higher order neural network (PHONN) and trigonometric higher order neural network (THONN) models. This study also uses UTHONN models to simulate foreign exchange rates and consumer price index with error approaching 10-6.

Sign in / Sign up

Export Citation Format

Share Document