RippleNet: A Recurrent Neural Network for Sharp Wave Ripple (SPW-R) Detection

AbstractHippocampal sharp wave ripples (SPW-R) have been identified as key bio-markers of important brain functions such as memory consolidation and decision making. Understanding their underlying mechanisms in healthy and pathological brain function and behaviour rely on accurate SPW-R detection. In this multidisciplinary study, we propose a novel, self-improving artificial intelligence (AI) detection method in the form of deep Recurrent Neural Networks (RNN) with Long Short-Term memory (LSTM) layers that can learn features of SPW-R events from raw, labeled input data. The approach contrasts conventional routines that typically relies on hand-crafted, heuristic feature extraction and often laborious manual curation. The algorithm is trained using supervised learning on hand-curated data sets with SPW-R events obtained under controlled conditions. The input to the algorithm is the local field potential (LFP), the low-frequency part of extracellularly recorded electric potentials from the CA1 region of the hippocampus. Its output predictions can be interpreted as time-varying probabilities of SPW-R events for the duration of the inputs. A simple thresholding applied to the output probabilities is found to identify times of SPW-R events with high precision. The non-causal, or bidirectional variant of the proposed algorithm demonstrates consistently better accuracy compared to the causal, or unidirectional counterpart. Reference implementations of the algorithm, named ‘RippleNet’, are open source, freely available, and implemented using a common open-source framework for neural networks () and can be easily incorporated into existing data analysis workflows for processing experimental data.

Download Full-text

CASM: A Deep-Learning Approach for Identifying Collective Action Events with Text and Image Data from Social Media

Sociological Methodology ◽

10.1177/0081175019860244 ◽

2019 ◽

Vol 49 (1) ◽

pp. 1-57 ◽

Cited By ~ 9

Author(s):

Han Zhang ◽

Jennifer Pan

Keyword(s):

Neural Networks ◽

Social Media ◽

Collective Action ◽

Short Term Memory ◽

Image Data ◽

Data Sets ◽

Event Analysis ◽

Text Data ◽

Out Of Sample ◽

Media Reports

Protest event analysis is an important method for the study of collective action and social movements and typically draws on traditional media reports as the data source. We introduce collective action from social media (CASM)—a system that uses convolutional neural networks on image data and recurrent neural networks with long short-term memory on text data in a two-stage classifier to identify social media posts about offline collective action. We implement CASM on Chinese social media data and identify more than 100,000 collective action events from 2010 to 2017 (CASM-China). We evaluate the performance of CASM through cross-validation, out-of-sample validation, and comparisons with other protest data sets. We assess the effect of online censorship and find it does not substantially limit our identification of events. Compared to other protest data sets, CASM-China identifies relatively more rural, land-related protests and relatively few collective action events related to ethnic and religious conflict.

Download Full-text

Efficacy of Feedforward and LSTM Neural Networks at Predicting and Gap Filling Coastal Ocean Timeseries: Oxygen, Nutrients, and Temperature

Frontiers in Marine Science ◽

10.3389/fmars.2021.637759 ◽

2021 ◽

Vol 8 ◽

Author(s):

Steefan Contractor ◽

Moninya Roughan

Keyword(s):

Neural Networks ◽

Decision Making ◽

Short Term Memory ◽

Data Sets ◽

Gap Filling ◽

Diverse Range ◽

Big Data Applications ◽

Free Data ◽

Before And After ◽

Ocean Environment

Ocean data timeseries are vital for a diverse range of stakeholders (ranging from government, to industry, to academia) to underpin research, support decision making, and identify environmental change. However, continuous monitoring and observation of ocean variables is difficult and expensive. Moreover, since oceans are vast, observations are typically sparse in spatial and temporal resolution. In addition, the hostile ocean environment creates challenges for collecting and maintaining data sets, such as instrument malfunctions and servicing, often resulting in temporal gaps of varying lengths. Neural networks (NN) have proven effective in many diverse big data applications, but few oceanographic applications have been tested using modern frameworks and architectures. Therefore, here we demonstrate a “proof of concept” neural network application using a popular “off-the-shelf” framework called “TensorFlow” to predict subsurface ocean variables including dissolved oxygen and nutrient (nitrate, phosphate, and silicate) concentrations, and temperature timeseries and show how these models can be used successfully for gap filling data products. We achieved a final prediction accuracy of over 96% for oxygen and temperature, and mean squared errors (MSE) of 2.63, 0.0099, and 0.78, for nitrates, phosphates, and silicates, respectively. The temperature gap-filling was done with an innovative contextual Long Short-Term Memory (LSTM) NN that uses data before and after the gap as separate feature variables. We also demonstrate the application of a novel dropout based approach to approximate the Bayesian uncertainty of these temperature predictions. This Bayesian uncertainty is represented in the form of 100 monte carlo dropout estimates of the two longest gaps in the temperature timeseries from a model with 25% dropout in the input and recurrent LSTM connections. Throughout the study, we present the NN training process including the tuning of the large number of NN hyperparameters which could pose as a barrier to uptake among researchers and other oceanographic data users. Our models can be scaled up and applied operationally to provide consistent, gap-free data to all data users, thus encouraging data uptake for data-based decision making.

Download Full-text

Improvement of Variant Adaptable LSTM Trained With Metaheuristic Algorithms for Healthcare Analysis

Research Anthology on Artificial Intelligence Applications in Security ◽

10.4018/978-1-7998-7705-9.ch048 ◽

2021 ◽

pp. 1031-1051

Author(s):

Tarik A. Rashid ◽

Mohammad K. Hassan ◽

Mokhtar Mohammadi ◽

Kym Fraser

Keyword(s):

Diabetes Mellitus ◽

Neural Networks ◽

Time Series Data ◽

Short Term Memory ◽

Learning Algorithm ◽

Back Propagation ◽

Machine Learning Techniques ◽

Series Data ◽

Data Sets ◽

Patients With Diabetes

Recently, the population of the world has increased along with health problems. Diabetes mellitus disease as an example causes issues to the health of many patients globally. The task of this chapter is to develop a dynamic and intelligent decision support system for patients with different diseases, and it aims at examining machine-learning techniques supported by optimization techniques. Artificial neural networks have been used in healthcare for several decades. Most research works utilize multilayer layer perceptron (MLP) trained with back propagation (BP) learning algorithm to achieve diabetes mellitus classification. Nonetheless, MLP has some drawbacks, such as, convergence, which can be slow; local minima can affect the training process. It is hard to scale and cannot be used with time series data sets. To overcome these drawbacks, long short-term memory (LSTM) is suggested, which is a more advanced form of recurrent neural networks. In this chapter, adaptable LSTM trained with two optimizing algorithms instead of the back propagation learning algorithm is presented. The optimization algorithms are biogeography-based optimization (BBO) and genetic algorithm (GA). Dataset is collected locally and another benchmark dataset is used as well. Finally, the datasets fed into adaptable models; LSTM with BBO (LSTMBBO) and LSTM with GA (LSTMGA) for classification purposes. The experimental and testing results are compared and they are promising. This system helps physicians and doctors to provide proper health treatment for patients with diabetes mellitus. Details of source code and implementation of our system can be obtained in the following link “https://github.com/hamakamal/LSTM.”

Download Full-text

Improvement of Variant Adaptable LSTM Trained With Metaheuristic Algorithms for Healthcare Analysis

Advances in Medical Technologies and Clinical Practice - Advanced Classification Techniques for Healthcare Analysis ◽

10.4018/978-1-5225-7796-6.ch006 ◽

2019 ◽

pp. 111-131 ◽

Cited By ~ 1

Author(s):

Tarik A. Rashid ◽

Mohammad K. Hassan ◽

Mokhtar Mohammadi ◽

Kym Fraser

Keyword(s):

Diabetes Mellitus ◽

Neural Networks ◽

Time Series Data ◽

Short Term Memory ◽

Learning Algorithm ◽

Back Propagation ◽

Machine Learning Techniques ◽

Series Data ◽

Data Sets ◽

Patients With Diabetes

Recently, the population of the world has increased along with health problems. Diabetes mellitus disease as an example causes issues to the health of many patients globally. The task of this chapter is to develop a dynamic and intelligent decision support system for patients with different diseases, and it aims at examining machine-learning techniques supported by optimization techniques. Artificial neural networks have been used in healthcare for several decades. Most research works utilize multilayer layer perceptron (MLP) trained with back propagation (BP) learning algorithm to achieve diabetes mellitus classification. Nonetheless, MLP has some drawbacks, such as, convergence, which can be slow; local minima can affect the training process. It is hard to scale and cannot be used with time series data sets. To overcome these drawbacks, long short-term memory (LSTM) is suggested, which is a more advanced form of recurrent neural networks. In this chapter, adaptable LSTM trained with two optimizing algorithms instead of the back propagation learning algorithm is presented. The optimization algorithms are biogeography-based optimization (BBO) and genetic algorithm (GA). Dataset is collected locally and another benchmark dataset is used as well. Finally, the datasets fed into adaptable models; LSTM with BBO (LSTMBBO) and LSTM with GA (LSTMGA) for classification purposes. The experimental and testing results are compared and they are promising. This system helps physicians and doctors to provide proper health treatment for patients with diabetes mellitus. Details of source code and implementation of our system can be obtained in the following link “https://github.com/hamakamal/LSTM.”

Download Full-text

Sentiment Recognition in Customer Reviews Using Deep Learning

Deep Learning and Neural Networks ◽

10.4018/978-1-7998-0414-7.ch077 ◽

2020 ◽

pp. 1383-1393

Author(s):

Vinay Kumar Jain ◽

Shishir Kumar ◽

Prabhat Kumar Mahanti

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Sentiment Analysis ◽

Text Classification ◽

Short Term Memory ◽

Machine Learning Techniques ◽

Data Sets ◽

Customer Reviews ◽

Weakly Supervised ◽

Comparison Of The Results

Deep learning has become popular in all aspect related to human judgments. Most machine learning techniques work well which includes text classification, text sequence learning, sentiment analysis, question-answer engine, etc. This paper has been focused on two objectives, firstly is to study the applicability of deep neural networks strategies for extracting sentiment present in social media data and customer reviews with effective training solutions. The second objective is to design deep networks that can be trained with these weakly supervised strategies in order to predict meaningful inferences. This paper presents the concept and steps of using deep learning for extraction sentiments from customer reviews. The extraction pulls out the features from the customer reviews using deep learning popular methods including Convolution neural networks (CNN) and Long Short-Term Memory (LSTM) architectures. The comparison of the results with tradition text classification method such as Naive Bayes(NB) and Support Vector Machine(SVM) using two data sets IMDB reviews and Amazon customer reviews have been presented. This work mainly focused on investigating the merit of using deep models for sentiment analysis in customer reviews.

Download Full-text

Expanding the Uses of Truck GPS Data in Freight Modeling and Planning Activities

Transportation Research Record Journal of the Transportation Research Board ◽

10.3141/2646-08 ◽

2017 ◽

Vol 2646 (1) ◽

pp. 68-76 ◽

Cited By ~ 6

Author(s):

Pedro Camargo ◽

Shuyao Hong ◽

Vladimir Livshits

Keyword(s):

Open Source ◽

Low Frequency ◽

The United States ◽

Time Of Day ◽

Gps Data ◽

Data Sets ◽

Trajectory Data ◽

Practical Applications ◽

Time Stamps ◽

Commercial Sources

Progress in practical applications of large, passively collected data sets is often hindered by the lack of appropriate analytical tools or the proprietary nature of applicable software. One of the most widely used data sources in the United States is truck GPS data that are commercially available from a few sources nationwide. Although many large GPS data sets are used in the development of tour-based truck models, the development of a fairly general approach to data analysis and processing that can be readily applied to various GPS data sets without need of proprietary software is still of interest. First, this paper presents a set of tools and techniques used to transform low-frequency truck GPS data available from commercial sources into complete trajectories on the network, that is, sequences of links constituting continuous paths traversed by each truck, with corresponding time stamps on each of the nodes. For this exercise, only open-source software was used, and the algorithm implementation was released as an open-source tool under a business-friendly license. Second, use of the truck GPS data was expanded beyond the standard extraction of trip matrices and estimation of tour models. Additional applications include select link analysis, time-of-day analysis, and trajectory data visualization.

Download Full-text

Human Activity Recognition using Fourier Transform Inspired Deep Learning Combination Model

International Journal of Sensors Wireless Communications and Control ◽

10.2174/2210327908666180727123657 ◽

2019 ◽

Vol 9 (1) ◽

pp. 16-31

Author(s):

Kyungkoo Jun

Keyword(s):

Fourier Transform ◽

Deep Learning ◽

Short Term Memory ◽

Window Size ◽

Sensor Data ◽

Data Sets ◽

Data Set ◽

Proposed Model ◽

Testing Data ◽

Labeling Scheme

Background & Objective: This paper proposes a Fourier transform inspired method to classify human activities from time series sensor data. Methods: Our method begins by decomposing 1D input signal into 2D patterns, which is motivated by the Fourier conversion. The decomposition is helped by Long Short-Term Memory (LSTM) which captures the temporal dependency from the signal and then produces encoded sequences. The sequences, once arranged into the 2D array, can represent the fingerprints of the signals. The benefit of such transformation is that we can exploit the recent advances of the deep learning models for the image classification such as Convolutional Neural Network (CNN). Results: The proposed model, as a result, is the combination of LSTM and CNN. We evaluate the model over two data sets. For the first data set, which is more standardized than the other, our model outperforms previous works or at least equal. In the case of the second data set, we devise the schemes to generate training and testing data by changing the parameters of the window size, the sliding size, and the labeling scheme. Conclusion: The evaluation results show that the accuracy is over 95% for some cases. We also analyze the effect of the parameters on the performance.

Download Full-text

Evaluation of Mixed Deep Neural Networks for Reverberant Speech Enhancement

Biomimetics ◽

10.3390/biomimetics5010001 ◽

2019 ◽

Vol 5 (1) ◽

pp. 1 ◽

Cited By ~ 1

Author(s):

Michelle Gutiérrez-Muñoz ◽

Astryd González-Salazar ◽

Marvin Coto-Jiménez

Keyword(s):

Neural Networks ◽

Short Term Memory ◽

Computational Cost ◽

Real Life ◽

Fixed Number ◽

Training Procedure ◽

Statistical Validation ◽

Significant Drop ◽

Training Time ◽

Important Solution

Speech signals are degraded in real-life environments, as a product of background noise or other factors. The processing of such signals for voice recognition and voice analysis systems presents important challenges. One of the conditions that make adverse quality difficult to handle in those systems is reverberation, produced by sound wave reflections that travel from the source to the microphone in multiple directions. To enhance signals in such adverse conditions, several deep learning-based methods have been proposed and proven to be effective. Recently, recurrent neural networks, especially those with long short-term memory (LSTM), have presented surprising results in tasks related to time-dependent processing of signals, such as speech. One of the most challenging aspects of LSTM networks is the high computational cost of the training procedure, which has limited extended experimentation in several cases. In this work, we present a proposal to evaluate the hybrid models of neural networks to learn different reverberation conditions without any previous information. The results show that some combinations of LSTM and perceptron layers produce good results in comparison to those from pure LSTM networks, given a fixed number of layers. The evaluation was made based on quality measurements of the signal’s spectrum, the training time of the networks, and statistical validation of results. In total, 120 artificial neural networks of eight different types were trained and compared. The results help to affirm the fact that hybrid networks represent an important solution for speech signal enhancement, given that reduction in training time is on the order of 30%, in processes that can normally take several days or weeks, depending on the amount of data. The results also present advantages in efficiency, but without a significant drop in quality.

Download Full-text