scholarly journals Sliding Window and Parallel LSTM with Attention and CNN for Sentence Alignment on Low-Resource Languages

2021 ◽  
Vol 30 (1) ◽  
pp. 97-121
Author(s):  
Tien-Ping Tan ◽  
Chai Kim Lim ◽  
Wan Rose Eliza Abdul Rahman
Keyword(s):  
Neural Network ◽  
Short Term Memory ◽  
Sliding Window ◽  
Classification Problem ◽  
Sentence Length ◽  
Alignment Algorithm ◽  
Text Corpus ◽  
English Sentence ◽  
Parallel Text ◽  
Window Approach

A parallel text corpus is an important resource for building a machine translation (MT) system. Existing resources such as translated documents, bilingual dictionaries, and translated subtitles are excellent resources for constructing parallel text corpus. A sentence alignment algorithm automatically aligns source sentences and target sentences because manual sentence alignment is resource-intensive. Over the years, sentence alignment approaches have improved from sentence length heuristics to statistical lexical models to deep neural networks. Solving the alignment problem as a classification problem is interesting as classification is the core of machine learning. This paper proposes a parallel long-short-term memory with attention and convolutional neural network (parallel LSTM+Attention+CNN) for classifying two sentences as parallel or non-parallel sentences. A sliding window approach is also proposed with the classifier to align sentences in the source and target languages. The proposed approach was compared with three classifiers, namely the feedforward neural network, CNN, and bi-directional LSTM. It is also compared with the BleuAlign sentence alignment system. The classification accuracy of these models was evaluated using Malay-English parallel text corpus and UN French-English parallel text corpus. The Malay-English sentence alignment performance was then evaluated using research documents and the very challenging Classical Malay-English document. The proposed classifier obtained more than 80% accuracy in categorizing parallel/non-parallel sentences with a model built using only five thousand training parallel sentences. It has a higher sentence alignment accuracy than other baseline systems.

scholarly journals Comparing End-to-End Machine Learning Methods for Spectra Classification

2021 ◽  
Vol 11 (23) ◽  
pp. 11520
Author(s):  
Yue Sun ◽  
Sandor Brockhauser ◽  
Péter Hegedűs
Keyword(s):  
Neural Network ◽  
Time Series ◽  
Short Term Memory ◽  
Classification Problem ◽  
High Energy ◽  
High Energy Density ◽  
Time Series Classification ◽  
Classification Models ◽  
Attention Model ◽  
End To End

In scientific research, spectroscopy and diffraction experimental techniques are widely used and produce huge amounts of spectral data. Learning patterns from spectra is critical during these experiments. This provides immediate feedback on the actual status of the experiment (e.g., time-resolved status of the sample), which helps guide the experiment. The two major spectral changes what we aim to capture are either the change in intensity distribution (e.g., drop or appearance) of peaks at certain locations, or the shift of those on the spectrum. This study aims to develop deep learning (DL) classification frameworks for one-dimensional (1D) spectral time series. In this work, we deal with the spectra classification problem from two different perspectives, one is a general two-dimensional (2D) space segmentation problem, and the other is a common 1D time series classification problem. We focused on the two proposed classification models under these two settings, the namely the end-to-end binned Fully Connected Neural Network (FCNN) with the automatically capturing weighting factors model and the convolutional SCT attention model. Under the setting of 1D time series classification, several other end-to-end structures based on FCNN, Convolutional Neural Network (CNN), ResNets, Long Short-Term Memory (LSTM), and Transformer were explored. Finally, we evaluated and compared the performance of these classification models based on the High Energy Density (HED) spectra dataset from multiple perspectives, and further performed the feature importance analysis to explore their interpretability. The results show that all the applied models can achieve 100% classification confidence, but the models applied under the 1D time series classification setting are superior. Among them, Transformer-based methods consume the least training time (0.449 s). Our proposed convolutional Spatial-Channel-Temporal (SCT) attention model uses 1.269 s, but its self-attention mechanism performed across spatial, channel, and temporal dimensions can suppress indistinguishable features better than others, and selectively focus on obvious features with high separability.

scholarly journals Capturing spike train temporal pattern with wavelet average coefficient for brain machine interface

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Shixian Wen ◽  
Allen Yin ◽  
Po-He Tseng ◽  
Laurent Itti ◽  
Mikhail A. Lebedev ◽  
...  
Keyword(s):  
Temporal Resolution ◽  
Short Term Memory ◽  
Sliding Window ◽  
Temporal Patterns ◽  
Spike Count ◽  
Neurological Injury ◽  
High Temporal Resolution ◽  
Long Time ◽  
Machine Interface ◽  
Window Approach

AbstractMotor brain machine interfaces (BMIs) directly link the brain to artificial actuators and have the potential to mitigate severe body paralysis caused by neurological injury or disease. Most BMI systems involve a decoder that analyzes neural spike counts to infer movement intent. However, many classical BMI decoders (1) fail to take advantage of temporal patterns of spike trains, possibly over long time horizons; (2) are insufficient to achieve good BMI performance at high temporal resolution, as the underlying Gaussian assumption of decoders based on spike counts is violated. Here, we propose a new statistical feature that represents temporal patterns or temporal codes of spike events with richer description—wavelet average coefficients (WAC)—to be used as decoder input instead of spike counts. We constructed a wavelet decoder framework by using WAC features with a sliding-window approach, and compared the resulting decoder against classical decoders (Wiener and Kalman family) and new deep learning based decoders ( Long Short-Term Memory) using spike count features. We found that the sliding-window approach boosts decoding temporal resolution, and using WAC features significantly improves decoding performance over using spike count features.

scholarly journals DoMars16k: A Diverse Dataset for Weakly Supervised Geomorphologic Analysis on Mars

2020 ◽  
Vol 12 (23) ◽  
pp. 3981
Author(s):  
Thorsten Wilhelm ◽  
Melina Geis ◽  
Jens Püttschneider ◽  
Timo Sievernich ◽  
Tobias Weber ◽  
...  
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Sliding Window ◽  
Landing Site ◽  
Martian Surface ◽  
Geomorphic Mapping ◽  
Markov Random ◽  
Weakly Supervised ◽  
Window Approach ◽  
Geomorphologic Analysis

Mapping planetary surfaces is an intricate task that forms the basis for many geologic, geomorphologic, and geographic studies of planetary bodies. In this work, we present a method to automate a specific type of planetary mapping, geomorphic mapping, taking machine learning as a basis. Additionally, we introduce a novel dataset, termed DoMars16k, which contains 16,150 samples of fifteen different landforms commonly found on the Martian surface. We use a convolutional neural network to establish a relation between Mars Reconnaissance Orbiter Context Camera images and the landforms of the dataset. Afterwards, we employ a sliding-window approach in conjunction with a Markov Random field smoothing to create maps in a weakly supervised fashion. Finally, we provide encouraging results and carry out automated geomorphological analyses of Jezero crater, the Mars2020 landing site, and Oxia Planum, the prospective ExoMars landing site.

scholarly journals Accelerometer-Based Human Activity Recognition for Patient Monitoring Using a Deep Neural Network

Sensors ◽  
2020 ◽  
Vol 20 (22) ◽  
pp. 6424
Author(s):  
Esther Fridriksdottir ◽  
Alberto G. Bonomi
Keyword(s):  
Neural Network ◽  
Deep Neural Network ◽  
Short Term Memory ◽  
Body Movement ◽  
Recovery Process ◽  
Classification Problem ◽  
Hospital Environment ◽  
Support Vector ◽  
Svm Classifier ◽  
Accelerometer Data

The objective of this study was to investigate the accuracy of a Deep Neural Network (DNN) in recognizing activities typical for hospitalized patients. A data collection study was conducted with 20 healthy volunteers (10 males and 10 females, age = 43 ± 13 years) in a simulated hospital environment. A single triaxial accelerometer mounted on the trunk was used to measure body movement and recognize six activity types: lying in bed, upright posture, walking, wheelchair transport, stair ascent and stair descent. A DNN consisting of a three-layer convolutional neural network followed by a long short-term memory layer was developed for this classification problem. Additionally, features were extracted from the accelerometer data to train a support vector machine (SVM) classifier for comparison. The DNN reached 94.52% overall accuracy on the holdout dataset compared to 83.35% of the SVM classifier. In conclusion, a DNN is capable of recognizing types of physical activity in simulated hospital conditions using data captured by a single tri-axial accelerometer. The method described may be used for continuous monitoring of patient activities during hospitalization to provide additional insights into the recovery process.

scholarly journals Personalized Predictive Models for Identifying Clinical Deterioration Using LSTM in Emergency Departments

2020 ◽  
Author(s):  
Amin Naemi ◽  
Thomas Schmidt ◽  
Marjan Mansourvar ◽  
Uffe Kock Wiil
Keyword(s):  
Neural Network ◽  
Predictive Models ◽  
Short Term Memory ◽  
Vital Signs ◽  
Illness Severity ◽  
Mortality And Morbidity ◽  
Proposed Model ◽  
Long Short Term Memory ◽  
Window Approach ◽  
Short Time

Early detection of deterioration at hospitals could be beneficial in terms of reducing mortality and morbidity rates and costs. In this paper, we present a model based on Long Short-Term Memory (LSTM) neural network used in deep learning to predict the illness severity of patients in advance. Hence, by predicting health severity, this model can be used to identify deteriorating patients. Our proposed model utilizes continuous monitored vital signs, including heart rate, respiratory rate, oxygen saturation, and blood pressure automatically collected from patients during hospitalization. In this study, a short-time prediction using a sliding window approach is applied. The performance of the proposed model was compared with the Multi-Layer Perceptron (MLP) neural network, a feedforward class of neural network, based on R2 score and Root Mean Square Error (RMSE) metrics. The results showed that the LSTM has a better performance and could predict the illness severity of patients more accurately.

Improving Sentiment Analysis using Hybrid Deep Learning Model

2020 ◽  
Vol 13 (4) ◽  
pp. 627-640 ◽  
Author(s):  
Avinash Chandra Pandey ◽  
Dharmveer Singh Rajpoot
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Sentiment Analysis ◽  
Classification Accuracy ◽  
Short Term Memory ◽  
Computational Cost ◽  
Extraction Process ◽  
Learning Model ◽  
Sentiment Classification ◽  
Deep Learning Model

Background: Sentiment analysis is a contextual mining of text which determines viewpoint of users with respect to some sentimental topics commonly present at social networking websites. Twitter is one of the social sites where people express their opinion about any topic in the form of tweets. These tweets can be examined using various sentiment classification methods to find the opinion of users. Traditional sentiment analysis methods use manually extracted features for opinion classification. The manual feature extraction process is a complicated task since it requires predefined sentiment lexicons. On the other hand, deep learning methods automatically extract relevant features from data hence; they provide better performance and richer representation competency than the traditional methods. Objective: The main aim of this paper is to enhance the sentiment classification accuracy and to reduce the computational cost. Method: To achieve the objective, a hybrid deep learning model, based on convolution neural network and bi-directional long-short term memory neural network has been introduced. Results: The proposed sentiment classification method achieves the highest accuracy for the most of the datasets. Further, from the statistical analysis efficacy of the proposed method has been validated. Conclusion: Sentiment classification accuracy can be improved by creating veracious hybrid models. Moreover, performance can also be enhanced by tuning the hyper parameters of deep leaning models.

scholarly journals Electricity Consumption Forecasting Based on a Bidirectional Long-Short-Term Memory Artificial Neural Network

2020 ◽  
Vol 13 (1) ◽  
pp. 104
Author(s):  
Dana-Mihaela Petroșanu ◽  
Alexandru Pîrjan
Keyword(s):  
Neural Network ◽  
Artificial Neural Network ◽  
Short Term Memory ◽  
Electricity Consumption ◽  
Short Term ◽  
Term Memory ◽  
Storage Room ◽  
Long Short Term Memory ◽  
Commercial Center ◽  
Business And Management

The accurate forecasting of the hourly month-ahead electricity consumption represents a very important aspect for non-household electricity consumers and system operators, and at the same time represents a key factor in what regards energy efficiency and achieving sustainable economic, business, and management operations. In this context, we have devised, developed, and validated within the paper an hourly month ahead electricity consumption forecasting method. This method is based on a bidirectional long-short-term memory (BiLSTM) artificial neural network (ANN) enhanced with a multiple simultaneously decreasing delays approach coupled with function fitting neural networks (FITNETs). The developed method targets the hourly month-ahead total electricity consumption at the level of a commercial center-type consumer and for the hourly month ahead consumption of its refrigerator storage room. The developed approach offers excellent forecasting results, highlighted by the validation stage’s results along with the registered performance metrics, namely 0.0495 for the root mean square error (RMSE) performance metric for the total hourly month-ahead electricity consumption and 0.0284 for the refrigerator storage room. We aimed for and managed to attain an hourly month-ahead consumed electricity prediction without experiencing a significant drop in the forecasting accuracy that usually tends to occur after the first two weeks, therefore achieving a reliable method that satisfies the contractor’s needs, being able to enhance his/her activity from the economic, business, and management perspectives. Even if the devised, developed, and validated forecasting solution for the hourly consumption targets a commercial center-type consumer, based on its accuracy, this solution can also represent a useful tool for other non-household electricity consumers due to its generalization capability.

Sign in / Sign up

Export Citation Format

Share Document