Estimation of trapezoidal-shaped overlapping nuclear pulse parameters based on a deep learning CNN-LSTM model

2021 ◽  
Vol 28 (3) ◽  
Author(s):  
Xing-Ke Ma ◽  
Hong-Quan Huang ◽  
Xiao Ji ◽  
He-Ye Dai ◽  
Jun-Hong Wu ◽  
...  
Keyword(s):  
Neural Network ◽  
Loss Function ◽  
Short Term Memory ◽  
Estimation Method ◽  
Pulse Signal ◽  
Learning Ability ◽  
Pulse Parameter ◽  
Data Set ◽  
Pulse Parameters ◽  
Model Training

The Long Short-Term Memory neural network (LSTM) has excellent learning ability for the time series of the nuclear pulse signal. It can accurately estimate the parameters (such as amplitude, time constant, etc.) of the digitally shaped nuclear pulse signal (especially the overlapping pulse signal). However, due to the large number of pulse sequences, the direct use of these sequences as samples to train the LSTM increases the complexity of the network, resulting in a lower training efficiency of the model. The convolution neural network (CNN) can effectively extract the sequence samples by using its unique convolution kernel structure, thus greatly reducing the number of sequence samples. Therefore, the CNN-LSTM deep neural network is used to estimate the parameters of overlapping pulse signals after digital trapezoidal shaping of exponential signals. Firstly, the estimation of the trapezoidal overlapping nuclear pulse is considered to be obtained after the superposition of multiple exponential nuclear pulses followed by trapezoidal shaping. Then, a data set containing multiple samples is set up; each sample is composed of the sequence of sampling values of the trapezoidal overlapping nuclear pulse and the set of shaping parameters of the exponential pulse before digital shaping. Secondly, the CNN is used to extract the abstract features of the training set in these samples, and then these abstract features are applied to the training of the LSTM model. In the training process, the pulse parameter set estimated by the present neural network is calculated by forward propagation. Thirdly, the loss function is used to calculate the loss value between the estimated pulse parameter set and the actual pulse parameter set. Finally, a gradient-based optimization algorithm is applied to update the weight by getting back the loss value together with the gradient of the loss function to the network, so as to realize the purpose of training the network. After model training was completed, the sampled values of the trapezoidal overlapping nuclear pulse were used as input to the CNN-LSTM model to obtain the required parameter set from the output of the CNN-LSTM model. The experimental results show that this method can effectively overcome the shortcomings of local convergence of traditional methods and greatly save the time of model training. At the same time, it can accurately estimate multiple trapezoidal overlapping pulses due to the wide width of the flat top, thus realizing the optimal estimation of nuclear pulse parameters in a global sense, which is a good pulse parameter estimation method.

scholarly journals A Dense Long Short-Term Memory Model for Enhancing the Imagery-Based Brain-Computer Interface

2021 ◽  
Vol 2021 ◽  
pp. 1-10
Author(s):  
Xiaofei Zhang ◽  
Tao Wang ◽  
Qi Xiong ◽  
Yina Guo
Keyword(s):  
Short Term Memory ◽  
Recognition Algorithm ◽  
Training Process ◽  
Short Term ◽  
Data Set ◽  
Static State ◽  
Term Memory ◽  
Long Short Term Memory ◽  
Model Training ◽  
The One

Imagery-based brain-computer interfaces (BCIs) aim to decode different neural activities into control signals by identifying and classifying various natural commands from electroencephalogram (EEG) patterns and then control corresponding equipment. However, several traditional BCI recognition algorithms have the “one person, one model” issue, where the convergence of the recognition model’s training process is complicated. In this study, a new BCI model with a Dense long short-term memory (Dense-LSTM) algorithm is proposed, which combines the event-related desynchronization (ERD) and the event-related synchronization (ERS) of the imagery-based BCI; model training and testing were conducted with its own data set. Furthermore, a new experimental platform was built to decode the neural activity of different subjects in a static state. Experimental evaluation of the proposed recognition algorithm presents an accuracy of 91.56%, which resolves the “one person one model” issue along with the difficulty of convergence in the training process.

Forecasting the momentum using customised loss function for financial series

2021 ◽  
Vol 14 (4) ◽  
pp. 702-713
Author(s):  
N. Prabakaran ◽  
Rajasekaran Palaniappan ◽  
R. Kannadasan ◽  
Satya Vinay Dudi ◽  
V. Sasidhar
Keyword(s):  
Neural Network ◽  
Time Series ◽  
Loss Function ◽  
Short Term Memory ◽  
Learning Algorithm ◽  
Financial Time Series ◽  
Storage Unit ◽  
Content Type ◽  
Financial Time ◽  
Unit Structure

PurposeWe propose a Machine Learning (ML) approach that will be trained from the available financial data and is able to gain the trends over the data and then uses the acquired knowledge for a more accurate forecasting of financial series. This work will provide a more precise results when weighed up to aged financial series forecasting algorithms. The LSTM Classic will be used to forecast the momentum of the Financial Series Index and also applied to its commodities. The network will be trained and evaluated for accuracy with various sizes of data sets, i.e. weekly historical data of MCX, GOLD, COPPER and the results will be calculated.Design/methodology/approachDesirable LSTM model for script price forecasting from the perspective of minimizing MSE. The approach which we have followed is shown below. (1) Acquire the Dataset. (2) Define your training and testing columns in the dataset. (3) Transform the input value using scalar. (4) Define the custom loss function. (5) Build and Compile the model. (6) Visualise the improvements in results.FindingsFinancial series is one of the very aged techniques where a commerce person would commerce financial scripts, make business and earn some wealth from these companies that vend a part of their business on trading manifesto. Forecasting financial script prices is complex tasks that consider extensive human–computer interaction. Due to the correlated nature of financial series prices, conventional batch processing methods like an artificial neural network, convolutional neural network, cannot be utilised efficiently for financial market analysis. We propose an online learning algorithm that utilises an upgraded of recurrent neural networks called long short-term memory Classic (LSTM). The LSTM Classic is quite different from normal LSTM as it has customised loss function in it. This LSTM Classic avoids long-term dependence on its metrics issues because of its unique internal storage unit structure, and it helps forecast financial time series. Financial Series Index is the combination of various commodities (time series). This makes Financial Index more reliable than the financial time series as it does not show a drastic change in its value even some of its commodities are affected. This work will provide a more precise results when weighed up to aged financial series forecasting algorithms.Originality/valueWe had built the customised loss function model by using LSTM scheme and have experimented on MCX index and as well as on its commodities and improvements in results are calculated for every epoch that we run for the whole rows present in the dataset. For every epoch we can visualise the improvements in loss. One more improvement that can be done to our model that the relationship between price difference and directional loss is specific to other financial scripts. Deep evaluations can be done to identify the best combination of these for a particular stock to obtain better results.

scholarly journals Multi-Regional Online Car-Hailing Order Quantity Forecasting Based on the Convolutional Neural Network

Information ◽  
2019 ◽  
Vol 10 (6) ◽  
pp. 193 ◽  
Author(s):  
Zihao Huang ◽  
Gang Huang ◽  
Zhijun Chen ◽  
Chaozhong Wu ◽  
Xiaofeng Ma ◽  
...  
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Travel Demand ◽  
Short Term Memory ◽  
Demand Forecasting ◽  
Image Feature ◽  
Support Vector ◽  
Data Set ◽  
Demand Distribution ◽  
Demand Forecasting Model

With the development of online cars, the demand for travel prediction is increasing in order to reduce the information asymmetry between passengers and drivers of online car-hailing. This paper proposes a travel demand forecasting model named OC-CNN based on the convolutional neural network to forecast the travel demand. In order to make full use of the spatial characteristics of the travel demand distribution, this paper meshes the prediction area and creates a travel demand data set of the graphical structure to preserve its spatial properties. Taking advantage of the convolutional neural network in image feature extraction, the historical demand data of the first twenty-five minutes of the entire region are used as a model input to predict the travel demand for the next five minutes. In order to verify the performance of the proposed method, one-month data from online car-hailing of the Chengdu Fourth Ring Road are used. The results show that the model successfully extracts the spatiotemporal features of the data, and the prediction accuracies of the proposed method are superior to those of the representative methods, including the Bayesian Ridge Model, Linear Regression, Support Vector Regression, and Long Short-Term Memory networks.

scholarly journals Deep Learning with Dynamically Weighted Loss Function for Sensor-Based Prognostics and Health Management

Sensors ◽  
2020 ◽  
Vol 20 (3) ◽  
pp. 723 ◽  
Author(s):  
Divish Rengasamy ◽  
Mina Jafari ◽  
Benjamin Rothwell ◽  
Xin Chen ◽  
Grazziela P. Figueredo
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Loss Function ◽  
Short Term Memory ◽  
Health Management ◽  
Remaining Useful Life ◽  
Loss Functions ◽  
Pressure System ◽  
Failure Data ◽  
Prognostic And Health Management

Deep learning has been employed to prognostic and health management of automotive and aerospace with promising results. Literature in this area has revealed that most contributions regarding deep learning is largely focused on the model’s architecture. However, contributions regarding improvement of different aspects in deep learning, such as custom loss function for prognostic and health management are scarce. There is therefore an opportunity to improve upon the effectiveness of deep learning for the system’s prognostics and diagnostics without modifying the models’ architecture. To address this gap, the use of two different dynamically weighted loss functions, a newly proposed weighting mechanism and a focal loss function for prognostics and diagnostics task are investigated. A dynamically weighted loss function is expected to modify the learning process by augmenting the loss function with a weight value corresponding to the learning error of each data instance. The objective is to force deep learning models to focus on those instances where larger learning errors occur in order to improve their performance. The two loss functions used are evaluated using four popular deep learning architectures, namely, deep feedforward neural network, one-dimensional convolutional neural network, bidirectional gated recurrent unit and bidirectional long short-term memory on the commercial modular aero-propulsion system simulation data from NASA and air pressure system failure data for Scania trucks. Experimental results show that dynamically-weighted loss functions helps us achieve significant improvement for remaining useful life prediction and fault detection rate over non-weighted loss function predictions.

A novel artificial neural network for flood forecasting based on deep learning encoder-decoder architecture

2020 ◽  
Author(s):  
Kangling Lin ◽  
Hua Chen ◽  
Chong-Yu Xu ◽  
Yanlai Zhou ◽  
Shenglian Guo
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Goodness Of Fit ◽  
Short Term Memory ◽  
Flood Forecasting ◽  
Learning Ability ◽  
Convolutional Network ◽  
Study Results ◽  
Decoder Architecture ◽  
Artificial Neural

<p>With the rapid growth of deep learning recently, artificial neural networks have been propelled to the forefront in flood forecasting via their end-to-end learning ability. Encoder-decoder architecture, as a novel deep feature extraction, which captures the inherent relationship of the data involved, has emerged in time sequence forecasting nowadays. As the advance of encoder-decoder architecture in sequence to sequence learning, it has been applied in many fields, such as machine translation, energy and environment. However, it is seldom used in hydrological modelling. In this study, a new neural network is developed to forecast flood based on the encoder-decoder architecture. There are two deep learning methods, including the Long Short-Term Memory (LSTM) network and Temporal Convolutional Network (TCN), selected as encoders respectively, while the LSTM was also chosen as the decoder, whose results are compared with those from the standard LSTM without using encoder-decoder architecture.</p><p>These models were trained and tested by using the hourly flood events data from 2009 to 2015 in Jianxi basin, China. The results indicated that the new neural flood forecasting networks based encoder-decoder architectures generally perform better than the standard LSTM, since they have better goodness-of-fit between forecasted and observed flood and produce the promising performance in multi-index assessment. The TCN as an encoder has better model stability and accuracy than LSTM as an encoder, especially in longer forecast periods and larger flood. The study results also show that the encoder-decoder architecture can be used as an effective deep learning solution in flood forecasting.</p><p></p>

scholarly journals Fall detection system based on BiLSTM neural network

2019 ◽  
Vol 7 (5) ◽  
pp. 01-12
Author(s):  
Biao YE ◽  
Lasheng Yu
Keyword(s):  
Neural Network ◽  
Short Term Memory ◽  
Detection System ◽  
Input Sequence ◽  
Fall Detection ◽  
Detection Algorithm ◽  
Training Data ◽  
Sensor Data ◽  
Data Set ◽  
Sequence Modeling

The purpose of this article is to analyze the characteristics of human fall behavior to design a fall detection system. The existing fall detection algorithms have problems such as poor adaptability, single function and difficulty in processing large data and strong randomness. Therefore, a long-term and short-term memory recurrent neural network is used to improve the effect of falling behavior detection by exploring the internal correlation between sensor data. Firstly, the serialization representation method of sensor data, training data and detection input data is designed. The BiLSTM network has the characteristics of strong ability to sequence modeling and it is used to reduce the dimension of the data required by the fall detection model. then, the BiLSTM training algorithm for fall detection and the BiLSTM-based fall detection algorithm convert the fall detection into the classification problem of the input sequence; finally, the BiLSTM-based fall detection system was implemented on the TensorFlow platform. The detection and analysis of system were carried out using a bionic experiment data set which mimics a fall. The experimental results verify that the system can effectively improve the accuracy of fall detection to 90.47%. At the same time, it can effectively detect the behavior of Near-falling, and help to take corresponding protective measures.

Variance error of multi-classification based anomaly detection for time series data

2020 ◽  
pp. 1-16
Author(s):  
Baoquan Wang ◽  
Tonghai Jiang ◽  
Xi Zhou ◽  
Bo Ma ◽  
Fan Zhao ◽  
...  
Keyword(s):  
Neural Network ◽  
Time Series ◽  
Anomaly Detection ◽  
Time Series Data ◽  
Short Term Memory ◽  
Computational Cost ◽  
Reconstruction Error ◽  
Detection Methods ◽  
Series Data ◽  
Data Set

For abnormal detection of time series data, the supervised anomaly detection methods require labeled data. While the range of outlier factors used by the existing semi-supervised methods varies with data, model and time, the threshold for determining abnormality is difficult to obtain, in addition, the computational cost of the way to calculate outlier factors from other data points in the data set is also very large. These make such methods difficult to practically apply. This paper proposes a framework named LSTM-VE which uses clustering combined with visualization method to roughly label normal data, and then uses the normal data to train long short-term memory (LSTM) neural network for semi-supervised anomaly detection. The variance error (VE) of the normal data category classification probability sequence is used as outlier factor. The framework enables anomaly detection based on deep learning to be practically applied and using VE avoids the shortcomings of existing outlier factors and gains a better performance. In addition, the framework is easy to expand because the LSTM neural network can be replaced with other classification models. Experiments on the labeled and real unlabeled data sets prove that the framework is better than replicator neural networks with reconstruction error (RNN-RS) and has good scalability as well as practicability.

scholarly journals Compliance with International Soft Law

2018 ◽  
Vol 9 (3) ◽  
pp. 1-15 ◽  
Author(s):  
Michael D'Rosario ◽  
John Zeleznikow
Keyword(s):  
Neural Network ◽  
Legal System ◽  
Goodness Of Fit ◽  
Predictive Accuracy ◽  
Estimation Method ◽  
System Structure ◽  
Soft Law ◽  
Network Modelling ◽  
Data Set

The present article considers the importance of legal system origin in compliance with ‘international soft law,' or normative provisions contained in non-binding texts. The study considers key economic and governance metrics on national acceptance an implementation of the first Basle accord. Employing a data set of 70 countries, the present study considers the role of market forces and bilateral and multi-lateral pressures on implementation of soft law. There is little known about the role of legal system structure-related variables as factors moderating the implementation of multi-lateral agreements and international soft law, such as the 1988 accord. The present study extends upon research within the extant literature by employing a novel estimation method, a neural network modelling technique, with multi-layer perceptron artificial neural network (MPANN). Consistent with earlier studies, the article identifies a significant and positive effect associated with democratic systems and the implementation of the Basle accord. However, extending upon traditional estimation techniques, the study identifies the significance of savings rates and government effectiveness in determining implementation. Notably, the method is able to achieve a superior goodness of fit and predictive accuracy in determining implementation.

scholarly journals Short-term power prediction of photovoltaic power station based on long short-term memory-back-propagation

2019 ◽  
Vol 15 (10) ◽  
pp. 155014771988313 ◽  
Author(s):  
Chi Hua ◽  
Erxi Zhu ◽  
Liang Kuang ◽  
Dechang Pi
Keyword(s):  
Neural Network ◽  
Short Term Memory ◽  
Back Propagation ◽  
Back Propagation Neural Network ◽  
Short Term ◽  
Data Set ◽  
Term Memory ◽  
Photovoltaic Power ◽  
Forecasting Method ◽  
Long Short Term Memory

Accurate prediction of the generation capacity of photovoltaic systems is fundamental to ensuring the stability of the grid and to performing scheduling arrangements correctly. In view of the temporal defect and the local minimum problem of back-propagation neural network, a forecasting method of power generation based on long short-term memory-back-propagation is proposed. On this basis, the traditional prediction data set is improved. According to the three traditional methods listed in this article, we propose a fourth method to improve the traditional photovoltaic power station short-term power generation prediction. Compared with the traditional method, the long short-term memory-back-propagation neural network based on the improved data set has a lower prediction error. At the same time, a horizontal comparison with the multiple linear regression and the support vector machine shows that the long short-term memory-back-propagation method has several advantages. Based on the long short-term memory-back-propagation neural network, the short-term forecasting method proposed in this article for generating capacity of photovoltaic power stations will provide a basis for dispatching plan and optimizing operation of power grid.

Sign in / Sign up

Export Citation Format

Share Document