Anaphoricity Determination of Anaphora Resolution in Uygur Pronoun Based on CNN-LSTM Model

2017 ◽  
Vol 16 (03) ◽  
pp. 1750016 ◽  
Author(s):  
Tian Shengwei ◽  
Li Dongbai ◽  
Yu Long ◽  
Feng Guanjun ◽  
Zhao Jianguo ◽  
...  
Keyword(s):  
Neural Network ◽  
Semantic Information ◽  
Short Term Memory ◽  
Feature Representation ◽  
Semantic Feature ◽  
Support Vector ◽  
Anaphora Resolution ◽  
Short Term ◽  
Long Short Term Memory

As a core subtask in anaphora resolution, anaphoricity determination has aroused the interest of researchers. However, in recent work, the influence caused by the deep semantic information and the context of the coreference elements have not been taken into account. In this paper, by combining the semantic feature of Uygur, we established a Convolutional Neural Network & Long Short-Term Memory (CNN_LSTM) model in determining the anaphoricity of Uygur pronoun. Firstly, the deep negative semantic feature representation is extracted via word2vec. Secondly, the shallow explicit feature representation of coreference elements is extracted by our system. Afterwards, two kinds of features are combined to recognize whether coreference element is referential or not. The results showed that the method we used can distinguish coreference element accurately, the ACC[Formula: see text] score is 90.18% and the ACC[Formula: see text] score is 89.93%, which are higher than ANN (Artificial Neural Network) and SVM (Support Vector Machine) respectively.

scholarly journals Predicting remaining useful life of rolling bearings based on deep feature representation and long short-term memory neural network

2018 ◽  
Vol 10 (12) ◽  
pp. 168781401881718 ◽  
Author(s):  
Wentao Mao ◽  
Jianliang He ◽  
Jiamei Tang ◽  
Yuan Li
Keyword(s):  
Neural Network ◽  
Life Prediction ◽  
Short Term Memory ◽  
Degradation Process ◽  
Remaining Useful Life ◽  
Feature Representation ◽  
Short Term ◽  
Term Memory ◽  
Useful Life ◽  
Long Short Term Memory

For bearing remaining useful life prediction problem, the traditional machine-learning-based methods are generally short of feature representation ability and incapable of adaptive feature extraction. Although deep-learning-based remaining useful life prediction methods proposed in recent years can effectively extract discriminative features for bearing fault, these methods tend to less consider temporal information of fault degradation process. To solve this problem, a new remaining useful life prediction approach based on deep feature representation and long short-term memory neural network is proposed in this article. First, a new criterion, named support vector data normalized correlation coefficient, is proposed to automatically divide the whole bearing life as normal state and fast degradation state. Second, deep features of bearing fault with good representation ability can be obtained from convolutional neural network by means of the marginal spectrum in Hilbert–Huang transform of raw vibration signals and health state label. Finally, by considering the temporal information of degradation process, these features are fed into a long short-term memory neural network to construct a remaining useful life prediction model. Experiments are conducted on bearing data sets of IEEE PHM Challenge 2012. The results show the significance of performance improvement of the proposed method in terms of predictive accuracy and numerical stability.

Categorizing Natural Language-Based Customer Satisfaction: An Implementation Method Using Support Vector Machine and Long Short-Term Memory Neural Network

2021 ◽  
Vol 13 (4) ◽  
Author(s):  
Ralph Sherwin A. Corpuz ◽  
Keyword(s):  
Neural Network ◽  
Support Vector Machine ◽  
Natural Language ◽  
Text Categorization ◽  
Short Term Memory ◽  
Support Vector ◽  
Feature Engineering ◽  
Short Term ◽  
Term Memory ◽  
Long Short Term Memory

Analyzing natural language-based Customer Satisfaction (CS) is a tedious process. This issue is practically true if one is to manually categorize large datasets. Fortunately, the advent of supervised machine learning techniques has paved the way toward the design of efficient categorization systems used for CS. This paper presents the feasibility of designing a text categorization model using two popular and robust algorithms – the Support Vector Machine (SVM) and Long Short-Term Memory (LSTM) Neural Network, in order to automatically categorize complaints, suggestions, feedbacks, and commendations. The study found that, in terms of training accuracy, SVM has best rating of 98.63% while LSTM has best rating of 99.32%. Such results mean that both SVM and LSTM algorithms are at par with each other in terms of training accuracy, but SVM is significantly faster than LSTM by approximately 35.47s. The training performance results of both algorithms are attributed on the limitations of the dataset size, high-dimensionality of both English and Tagalog languages, and applicability of the feature engineering techniques used. Interestingly, based on the results of actual implementation, both algorithms are found to be 100% effective in accurately predicting the correct CS categories. Hence, the extent of preference between the two algorithms boils down on the available dataset and the skill in optimizing these algorithms through feature engineering techniques and in implementing them toward actual text categorization applications.

scholarly journals MULTI-TEMPORAL LAND COVER CLASSIFICATION WITH LONG SHORT-TERM MEMORY NEURAL NETWORKS

2017 ◽  
Vol XLII-1/W1 ◽  
pp. 551-558 ◽  
Author(s):  
M. Rußwurm ◽  
M. Körner
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Land Cover ◽  
Short Term Memory ◽  
Land Cover Classification ◽  
Support Vector ◽  
Short Term ◽  
Term Memory ◽  
Classification Techniques ◽  
Long Short Term Memory

<i>Land cover classification (LCC)</i> is a central and wide field of research in earth observation and has already put forth a variety of classification techniques. Many approaches are based on classification techniques considering observation at certain points in time. However, some land cover classes, such as crops, change their spectral characteristics due to environmental influences and can thus not be monitored effectively with classical mono-temporal approaches. Nevertheless, these temporal observations should be utilized to benefit the classification process. After extensive research has been conducted on modeling temporal dynamics by spectro-temporal profiles using vegetation indices, we propose a deep learning approach to utilize these temporal characteristics for classification tasks. In this work, we show how <i>long short-term memory</i> (LSTM) neural networks can be employed for crop identification purposes with SENTINEL 2A observations from large study areas and label information provided by local authorities. We compare these temporal neural network models, <i>i.e.</i>, LSTM and <i>recurrent neural network</i> (RNN), with a classical non-temporal <i>convolutional neural network</i> (CNN) model and an additional <i>support vector machine</i> (SVM) baseline. With our rather straightforward LSTM variant, we exceeded state-of-the-art classification performance, thus opening promising potential for further research.

scholarly journals Ekstraksi Relasi Antar Entitas di Bahasa Indonesia Menggunakan Neural Network

2021 ◽  
Vol 3 (02) ◽  
pp. 49-54
Author(s):  
Ananta Tio Putra ◽  
Eunike Kardinata ◽  
Hartarto Junaedi ◽  
Francisca Chandra ◽  
Joan Santoso
Keyword(s):  
Neural Network ◽  
Recurrent Neural Network ◽  
Short Term Memory ◽  
Relation Extraction ◽  
Support Vector ◽  
Short Term ◽  
Tree Kernel ◽  
Kernel Support Vector Machine ◽  
Long Short Term Memory ◽  
Bahasa Indonesia

Dengan perkembangan zaman yang begitu pesat, berdampak pada perkembangan data pula. Salah satu bentuk data yang paling banyak saat ini berupa data tekstual seperti artikel sederhana maupun dokumen lain yang terdapat di internet. Agar data tekstual tersebut dapat dimengerti dan dimanfaatkan dengan baik oleh manusia, maka perlu di proses dan disederhanakan agar menjadi informasi yang ringkas dan jelas. Oleh karena itu, semakin berkembang pula penelitian dalam bidang Information Extraction (IE) dan salah satu contoh penelitian di IE adalah Relation Extraction (RE). Penelitian RE sudah banyak dilakukan terutama pada Bahasa Inggris dimana resourcenya sudah termasuk banyak. Metode yang digunakan pun bermacam-macam seperti kernel, tree kernel, support vector machine, long short-term memory, convulution recurrent neural network, dan lain sebagainya. Pada penelitian kali ini adalah penelitian RE pada Bahasa Indonesia dengan menggunakan metode convulution recurrent neural network yang sudah dipergunakan untuk RE Bahasa Inggris. Dataset yang digunakan pada penelitian ini adalah dataset Bahasa Indonesia yang berasal dari file xml wikipedia. File xml wikipedia ini kemudian diproses sehingga menghasilkan dataset seperti yang digunakan pada CRNN dalam Bahasa inggris yaitu dalam format SemEval-2 Task 8. Uji coba dilakukan dengan berbagai macam perbandingan data training dan testing yaitu 80:20, 70:30, dan 60:40. Selain itu, parameter pooling untuk CRNN yang digunakan ada dua macam yaitu ‘att’ dan ‘max’. Dari uji coba yang dilakukan, hasil yang didapatkan adalah bervariasi mulai dari mendekati maupun lebih baik bila dibandingkan dengan CRNN dengan menggunakan dataset Bahasa inggris sehingga dapat disimpulkan bahwa dengan CRNN ini bisa digunakan untuk proses RE pada Bahasa Indonesia apabila dataset yang digunakan sesuai dengan penelitian sebelumnya.

scholarly journals Deep neural network-based classification of cardiotocograms outperformed conventional algorithms

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Jun Ogasawara ◽  
Satoru Ikenoue ◽  
Hiroko Yamamoto ◽  
Motoshige Sato ◽  
Yoshifumi Kasuga ◽  
...  
Keyword(s):  
Neural Network ◽  
Network Model ◽  
Neural Network Model ◽  
Deep Neural Network ◽  
Short Term Memory ◽  
Fetal Heart ◽  
Support Vector ◽  
Short Term ◽  
Term Memory ◽  
Long Short Term Memory

AbstractCardiotocography records fetal heart rates and their temporal relationship to uterine contractions. To identify high risk fetuses, obstetricians inspect cardiotocograms (CTGs) by eye. Therefore, CTG traces are often interpreted differently among obstetricians, resulting in inappropriate interventions. However, few studies have focused on quantitative and nonbiased algorithms for CTG evaluation. In this study, we propose a newly constructed deep neural network model (CTG-net) to detect compromised fetal status. CTG-net consists of three convolutional layers that extract temporal patterns and interrelationships between fetal heart rate and uterine contraction signals. We aimed to classify the abnormal group (umbilical artery pH < 7.20 or Apgar score at 1 min < 7) and the normal group from CTG data. We evaluated the performance of the CTG-net with the F1 score and compared it with conventional algorithms, namely, support vector machine and k-means clustering, and another deep neural network model, long short-term memory. CTG-net showed the area under the receiver operating characteristic curve of 0.73 ± 0.04, which was significantly higher than that of long short-term memory. CTG-net, a quantitative and automated diagnostic aid system, enables early intervention for putatively abnormal fetuses, resulting in a reduction in the number of cases of hypoxic injury.

scholarly journals Dynamic Space Allocation Based on Internal Demand for Optimizing Release of Shared Parking

Sensors ◽  
2021 ◽  
Vol 22 (1) ◽  
pp. 235
Author(s):  
Shuo-Yan Chou ◽  
Anindhita Dewabharata ◽  
Ferani Eva Zulvia
Keyword(s):  
Neural Network ◽  
Recurrent Neural Network ◽  
Short Term Memory ◽  
Support Vector ◽  
Time Data ◽  
Short Term ◽  
Term Memory ◽  
Parking Lots ◽  
Parking System ◽  
Long Short Term Memory

The size of cities has been continuously increasing because of urbanization. The number of public and private transportation vehicles is rapidly increasing, thus resulting in traffic congestion, traffic accidents, and environmental pollution. Although major cities have undergone considerable development in terms of transportation infrastructure, problems caused by a high number of moving vehicles cannot be completely resolved through the expansion of streets and facilities. This paper proposes a solution for the parking problem in cities that entails a shared parking system. The primary concept of the proposed shared parking system is to release parking lots that are open to specific groups for public usage without overriding personal usage. Open-to-specific-groups parking lots consist of parking spaces provided for particular people, such as parking buildings at universities for teachers, staff, and students. The proposed shared parking system comprises four primary steps: collecting and preprocessing data by using an Internet of Things system, predicting internal demand by using a recurrent neural network algorithm, releasing several unoccupied parking lots based on prediction results, and continuously updating the real-time data to improve future internal usage prediction. Data collection and data forecasting are performed to ensure that the system does not override personal usage. This study applied several forecasting algorithms, including seasonal ARIMA, support vector regression, multilayer perceptron, convolutional neural network, long short-term memory recurrent neural network with a many-to-one structure, and long short-term memory recurrent neural network with a many-to-many structure. The proposed system was evaluated using artificial and real datasets. Results show that the recurrent neural network with the many-to-many structure generates the most accurate prediction. Furthermore, the proposed shared parking system was evaluated for some scenarios in which different numbers of parking spaces were released. Simulation results show that the proposed shared parking system can provide parking spaces for public usage without overriding personal usage. Moreover, this system can generate new income for parking management and/or parking lot owners.

scholarly journals Stock Index Prices Prediction via Temporal Pattern Attention and Long-Short-Term Memory

2020 ◽  
Vol 2020 ◽  
pp. 1-7
Author(s):  
Xiaolu Wei ◽  
Binbin Lei ◽  
Hongbing Ouyang ◽  
Qiufeng Wu
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Time Series ◽  
Temporal Pattern ◽  
Short Term Memory ◽  
Stock Index ◽  
Support Vector ◽  
Short Term ◽  
Term Memory ◽  
Long Short Term Memory

This study attempts to predict stock index prices using multivariate time series analysis. The study’s motivation is based on the notion that datasets of stock index prices involve weak periodic patterns, long-term and short-term information, for which traditional approaches and current neural networks such as Autoregressive models and Support Vector Machine (SVM) may fail. This study applied Temporal Pattern Attention and Long-Short-Term Memory (TPA-LSTM) for prediction to overcome the issue. The results show that stock index prices prediction through the TPA-LSTM algorithm could achieve better prediction performance over traditional deep neural networks, such as recurrent neural network (RNN), convolutional neural network (CNN), and long and short-term time series network (LSTNet).

scholarly journals Improving Monsoon Precipitation Prediction Using Combined Convolutional and Long Short Term Memory Neural Network

Water ◽  
2019 ◽  
Vol 11 (5) ◽  
pp. 977 ◽  
Author(s):  
Qinghua Miao ◽  
Baoxiang Pan ◽  
Hao Wang ◽  
Kuolin Hsu ◽  
Soroosh Sorooshian
Keyword(s):  
Neural Network ◽  
Short Term Memory ◽  
Precipitation Forecast ◽  
Support Vector ◽  
Short Term ◽  
Term Memory ◽  
Precipitation Prediction ◽  
Downscaling Method ◽  
Forecast Lead Time ◽  
Long Short Term Memory

Precipitation downscaling is widely employed for enhancing the resolution and accuracy of precipitation products from general circulation models (GCMs). In this study, we propose a novel statistical downscaling method to foster GCMs’ precipitation prediction resolution and accuracy for the monsoon region. We develop a deep neural network composed of a convolution and Long Short Term Memory (LSTM) recurrent module to estimate precipitation based on well-resolved atmospheric dynamical fields. The proposed model is compared against the GCM precipitation product and classical downscaling methods in the Xiangjiang River Basin in South China. Results show considerable improvement compared to the European Centre for Medium-Range Weather Forecasts (ECMWF)-Interim reanalysis precipitation. Also, the model outperforms benchmark downscaling approaches, including (1) quantile mapping, (2) the support vector machine, and (3) the convolutional neural network. To test the robustness of the model and its applicability in practical forecasting, we apply the trained network for precipitation prediction forced by retrospective forecasts from the ECMWF model. Compared to the ECMWF precipitation forecast, our model makes better use of the resolved dynamical field for more accurate precipitation prediction at lead times from 1 day up to 2 weeks. This superiority decreases with the forecast lead time, as the GCM’s skill in predicting atmospheric dynamics is diminished by the chaotic effect. Finally, we build a distributed hydrological model and force it with different sources of precipitation inputs. Hydrological simulation forced with the neural network precipitation estimation shows significant advantage over simulation forced with the original ERA-Interim precipitation (with NSE value increases from 0.06 to 0.64), and the performance is only slightly worse than the observed precipitation forced simulation (NSE = 0.82). This further proves the value of the proposed downscaling method, and suggests its potential for hydrological forecasts.

scholarly journals Attention Neural Network for Biomedical Word Sense Disambiguation

2022 ◽  
Vol 2022 ◽  
pp. 1-14
Author(s):  
Chun-Xiang Zhang ◽  
Shu-Yang Pang ◽  
Xue-Yao Gao ◽  
Jia-Qi Lu ◽  
Bo Yu
Keyword(s):  
Neural Network ◽  
Semantic Information ◽  
Short Term Memory ◽  
Word Sense Disambiguation ◽  
Semantic Category ◽  
Word Sense ◽  
Short Term ◽  
Part Of Speech ◽  
Sense Disambiguation ◽  
Long Short Term Memory

In order to improve the disambiguation accuracy of biomedical words, this paper proposes a disambiguation method based on the attention neural network. The biomedical word is viewed as the center. Morphology, part of speech, and semantic information from 4 adjacent lexical units are extracted as disambiguation features. The attention layer is used to generate a feature matrix. Average asymmetric convolutional neural networks (Av-ACNN) and bidirectional long short-term memory (Bi-LSTM) networks are utilized to extract features. The softmax function is applied to determine the semantic category of the biomedical word. At the same time, CNN, LSTM, and Bi-LSTM are applied to biomedical WSD. MSH corpus is adopted to optimize CNN, LSTM, Bi-LSTM, and the proposed method and testify their disambiguation performance. Experimental results show that the average disambiguation accuracy of the proposed method is improved compared with CNN, LSTM, and Bi-LSTM. The average disambiguation accuracy of the proposed method achieves 91.38%.

Sign in / Sign up

Export Citation Format

Share Document