scholarly journals Using BiLSTM Networks for Context-Aware Deep Sensitivity Labelling on Conversational Data

2020 ◽  
Vol 10 (24) ◽  
pp. 8924
Author(s):  
Antreas Pogiatzis ◽  
Georgios Samakovitis
Keyword(s):  
Conditional Random Fields ◽  
Short Term Memory ◽  
Model Performance ◽  
Design Feature ◽  
Sensitive Information ◽  
Context Aware ◽  
Improve Model ◽  
Unseen Data ◽  
Memory Network ◽  
Series Of Experiments

Information privacy is a critical design feature for any exchange system, with privacy-preserving applications requiring, most of the time, the identification and labelling of sensitive information. However, privacy and the concept of “sensitive information” are extremely elusive terms, as they are heavily dependent upon the context they are conveyed in. To accommodate such specificity, we first introduce a taxonomy of four context classes to categorise relationships of terms with their textual surroundings by meaning, interaction, precedence, and preference. We then propose a predictive context-aware model based on a Bidirectional Long Short Term Memory network with Conditional Random Fields (BiLSTM + CRF) to identify and label sensitive information in conversational data (multi-class sensitivity labelling). We train our model on a synthetic annotated dataset of real-world conversational data categorised in 13 sensitivity classes that we derive from the P3P standard. We parameterise and run a series of experiments featuring word and character embeddings and introduce a set of auxiliary features to improve model performance. Our results demonstrate that the BiLSTM + CRF model architecture with BERT embeddings and WordShape features is the most effective (F1 score 96.73%). Evaluation of the model is conducted under both temporal and semantic contexts, achieving a 76.33% F1 score on unseen data and outperforms Google’s Data Loss Prevention (DLP) system on sensitivity labelling tasks.

scholarly journals Stock Trend Prediction Using Deep Learning Approach on Technical Indicator and Industrial Specific Information

Information ◽  
2021 ◽  
Vol 12 (6) ◽  
pp. 250
Author(s):  
Kittisak Prachyachuwong ◽  
Peerapon Vateekul
Keyword(s):  
Deep Learning ◽  
Short Term Memory ◽  
Predictive Accuracy ◽  
Model Performance ◽  
Specific Information ◽  
Trend Prediction ◽  
Textual Information ◽  
Economic News ◽  
Technical Indicator ◽  
Memory Network

A stock trend prediction has been in the spotlight from the past to the present. Fortunately, there is an enormous amount of information available nowadays. There were prior attempts that have tried to forecast the trend using textual information; however, it can be further improved since they relied on fixed word embedding, and it depends on the sentiment of the whole market. In this paper, we propose a deep learning model to predict the Thailand Futures Exchange (TFEX) with the ability to analyze both numerical and textual information. We have used Thai economic news headlines from various online sources. To obtain better news sentiment, we have divided the headlines into industry-specific indexes (also called “sectors”) to reflect the movement of securities of the same fundamental. The proposed method consists of Long Short-Term Memory Network (LSTM) and Bidirectional Encoder Representations from Transformers (BERT) architectures to predict daily stock market activity. We have evaluated model performance by considering predictive accuracy and the returns obtained from the simulation of buying and selling. The experimental results demonstrate that enhancing both numerical and textual information of each sector can improve prediction performance and outperform all baselines.

scholarly journals A Destination Prediction Network Based on Spatiotemporal Data for Bike-Sharing

Complexity ◽  
2019 ◽  
Vol 2019 ◽  
pp. 1-14 ◽  
Author(s):  
Jian Jiang ◽  
Fei Lin ◽  
Jin Fan ◽  
Hang Lv ◽  
Jia Wu
Keyword(s):  
Neural Network ◽  
Short Term Memory ◽  
Sharing Economy ◽  
Spatiotemporal Data ◽  
Low Carbon ◽  
Bike Sharing ◽  
Memory Network ◽  
User Demand ◽  
Series Of Experiments ◽  
Fully Connected

Bike-sharing is a new low-carbon and environment-friendly mode of public transport based on the “sharing economy”. Since 2017, the bike-sharing market has boomed in China’s major cities. Bikes equipped with GPS transmitters are docked along sidewalks that can be easily accessed through smartphone apps. However, this new form of transport has also led to problems, such as illegal parking, vandalism, and theft, each of which presents a major administrative challenge. Further, imbalances in user demand and bike availability need to be overcome to ensure a convenient, flexible service for customers. Hence, predicting a cyclist’s destination could be of great importance to shared-bike operators. In this paper, we propose an innovative deep learning model to predict the most probable destination for each user. The model, called destination prediction network based on spatiotemporal data (DPNst), comprises three steps. First, the data is preprocessed and a pool of likely candidate destinations is generated based on frequent item mining. This candidate set is then used to build the DPNst model: a long short-term memory network learns the user’s behavior; a convolutional neural network learns the spatial relationships between the origin and the candidate destinations; and a fully connected neural network learns the external features. In the final step, DPNst dynamically aggregates the output of the three neural networks based on the given data and generates the predictions. In a series of experiments on real-world stationless bike-sharing data, DPNst returned an F1 score of 42.71% and demonstrated better performance overall than the compared baselines.

scholarly journals Leveraging Label Information in a Knowledge-Driven Approach for Rolling-Element Bearings Remaining Useful Life Prediction

Energies ◽  
2021 ◽  
Vol 14 (8) ◽  
pp. 2163
Author(s):  
Tarek Berghout ◽  
Mohamed Benbouzid ◽  
Leïla-Hayet Mouss
Keyword(s):  
Transfer Learning ◽  
Short Term Memory ◽  
Remaining Useful Life ◽  
Accelerated Life Tests ◽  
Learning Path ◽  
Accelerated Life ◽  
Unseen Data ◽  
Label Information ◽  
Life Tests ◽  
Ill Posed

Since bearing deterioration patterns are difficult to collect from real, long lifetime scenarios, data-driven research has been directed towards recovering them by imposing accelerated life tests. Consequently, insufficiently recovered features due to rapid damage propagation seem more likely to lead to poorly generalized learning machines. Knowledge-driven learning comes as a solution by providing prior assumptions from transfer learning. Likewise, the absence of true labels was able to create inconsistency related problems between samples, and teacher-given label behaviors led to more ill-posed predictors. Therefore, in an attempt to overcome the incomplete, unlabeled data drawbacks, a new autoencoder has been designed as an additional source that could correlate inputs and labels by exploiting label information in a completely unsupervised learning scheme. Additionally, its stacked denoising version seems to more robustly be able to recover them for new unseen data. Due to the non-stationary and sequentially driven nature of samples, recovered representations have been fed into a transfer learning, convolutional, long–short-term memory neural network for further meaningful learning representations. The assessment procedures were benchmarked against recent methods under different training datasets. The obtained results led to more efficiency confirming the strength of the new learning path.

scholarly journals Predicting Future Occurrence of Acute Hypotensive Episodes Using Noninvasive and Invasive Features

2021 ◽  
Vol 186 (Supplement_1) ◽  
pp. 445-451
Author(s):  
Yifei Sun ◽  
Navid Rashedi ◽  
Vikrant Vaze ◽  
Parikshit Shah ◽  
Ryan Halter ◽  
...  
Keyword(s):  
Machine Learning ◽  
Support Vector Machine ◽  
Real World ◽  
Short Term Memory ◽  
Model Performance ◽  
Learning Technologies ◽  
Machine Learning Algorithms ◽  
Support Vector ◽  
K Nearest Neighbor ◽  
Continuous Map

ABSTRACT Introduction Early prediction of the acute hypotensive episode (AHE) in critically ill patients has the potential to improve outcomes. In this study, we apply different machine learning algorithms to the MIMIC III Physionet dataset, containing more than 60,000 real-world intensive care unit records, to test commonly used machine learning technologies and compare their performances. Materials and Methods Five classification methods including K-nearest neighbor, logistic regression, support vector machine, random forest, and a deep learning method called long short-term memory are applied to predict an AHE 30 minutes in advance. An analysis comparing model performance when including versus excluding invasive features was conducted. To further study the pattern of the underlying mean arterial pressure (MAP), we apply a regression method to predict the continuous MAP values using linear regression over the next 60 minutes. Results Support vector machine yields the best performance in terms of recall (84%). Including the invasive features in the classification improves the performance significantly with both recall and precision increasing by more than 20 percentage points. We were able to predict the MAP with a root mean square error (a frequently used measure of the differences between the predicted values and the observed values) of 10 mmHg 60 minutes in the future. After converting continuous MAP predictions into AHE binary predictions, we achieve a 91% recall and 68% precision. In addition to predicting AHE, the MAP predictions provide clinically useful information regarding the timing and severity of the AHE occurrence. Conclusion We were able to predict AHE with precision and recall above 80% 30 minutes in advance with the large real-world dataset. The prediction of regression model can provide a more fine-grained, interpretable signal to practitioners. Model performance is improved by the inclusion of invasive features in predicting AHE, when compared to predicting the AHE based on only the available, restricted set of noninvasive technologies. This demonstrates the importance of exploring more noninvasive technologies for AHE prediction.

scholarly journals Small Floating Target Detection Method Based on Chaotic Long Short-Term Memory Network

2021 ◽  
Vol 9 (6) ◽  
pp. 651
Author(s):  
Yan Yan ◽  
Hongyan Xing
Keyword(s):  
High Frequency ◽  
Short Term Memory ◽  
Detection Method ◽  
Low Frequency ◽  
Sea Clutter ◽  
Short Term ◽  
Term Memory ◽  
Memory Network ◽  
Long Short Term Memory ◽  
Small Targets

In order for the detection ability of floating small targets in sea clutter to be improved, on the basis of the complete ensemble empirical mode decomposition (CEEMD) algorithm, the high-frequency parts and low-frequency parts are determined by the energy proportion of the intrinsic mode function (IMF); the high-frequency part is denoised by wavelet packet transform (WPT), whereas the denoised high-frequency IMFs and low-frequency IMFs reconstruct the pure sea clutter signal together. According to the chaotic characteristics of sea clutter, we proposed an adaptive training timesteps strategy. The training timesteps of network were determined by the width of embedded window, and the chaotic long short-term memory network detection was designed. The sea clutter signals after denoising were predicted by chaotic long short-term memory (LSTM) network, and small target signals were detected from the prediction errors. The experimental results showed that the CEEMD-WPT algorithm was consistent with the target distribution characteristics of sea clutter, and the denoising performance was improved by 33.6% on average. The proposed chaotic long- and short-term memory network, which determines the training step length according to the width of embedded window, is a new detection method that can accurately detect small targets submerged in the background of sea clutter.

Sign in / Sign up

Export Citation Format

Share Document