scholarly journals PQPS: Prior-Art Query-Based Patent Summarizer Using RBM and Bi-LSTM

2021 ◽  
Vol 2021 ◽  
pp. 1-19
Author(s):  
Girthana Kumaravel ◽  
Swamynathan Sankaranarayanan
Keyword(s):  
Short Term Memory ◽  
Knowledge Bases ◽  
Smart Device ◽  
Mismatch Problem ◽  
Prior Art ◽  
Retrieval Rate ◽  
Novel Approach ◽  
Manual Processing ◽  
Long Short Term Memory ◽  
Ease Of Access

A prior-art search on patents ascertains the patentability constraints of the invention through an organized review of prior-art document sources. This search technique poses challenges because of the inherent vocabulary mismatch problem. Manual processing of every retrieved relevant patent in its entirety is a tedious and time-consuming job that demands automated patent summarization for ease of access. This paper employs deep learning models for summarization as they take advantage of the massive dataset present in the patents to improve the summary coherence. This work presents a novel approach of patent summarization named PQPS: prior-art query-based patent summarizer using restricted Boltzmann machine (RBM) and bidirectional long short-term memory (Bi-LSTM) models. The PQPS also addresses the vocabulary mismatch problem through query expansion with knowledge bases such as domain ontology and WordNet. It further enhances the retrieval rate through topic modeling and bibliographic coupling of citations. The experiments analyze various interlinked smart device patent sample sets. The proposed PQPS demonstrates that retrievability increases both in extractive and abstractive summaries.

scholarly journals A Hybrid Spoken Language Processing System for Smart Device Troubleshooting

Electronics ◽  
2019 ◽  
Vol 8 (6) ◽  
pp. 681 ◽  
Author(s):  
Praveen Edward James ◽  
Hou Kit Mun ◽  
Chockalingam Aravind Vaithilingam
Keyword(s):  
Language Processing ◽  
Short Term Memory ◽  
Processing System ◽  
Memory Cell ◽  
Spoken Language ◽  
Smart Device ◽  
Short Term ◽  
Language Processor ◽  
Spoken Language Processing ◽  
Long Short Term Memory

The purpose of this work is to develop a spoken language processing system for smart device troubleshooting using human-machine interaction. This system combines a software Bidirectional Long Short Term Memory Cell (BLSTM)-based speech recognizer and a hardware LSTM-based language processor for Natural Language Processing (NLP) using the serial RS232 interface. Mel Frequency Cepstral Coefficient (MFCC)-based feature vectors from the speech signal are directly input into a BLSTM network. A dropout layer is added to the BLSTM layer to reduce over-fitting and improve robustness. The speech recognition component is a combination of an acoustic modeler, pronunciation dictionary, and a BLSTM network for generating query text, and executes in real time with an 81.5% Word Error Rate (WER) and average training time of 45 s. The language processor comprises a vectorizer, lookup dictionary, key encoder, Long Short Term Memory Cell (LSTM)-based training and prediction network, and dialogue manager, and transforms query intent to generate response text with a processing time of 0.59 s, 5% hardware utilization, and an F1 score of 95.2%. The proposed system has a 4.17% decrease in accuracy compared with existing systems. The existing systems use parallel processing and high-speed cache memories to perform additional training, which improves the accuracy. However, the performance of the language processor has a 36.7% decrease in processing time and 50% decrease in hardware utilization, making it suitable for troubleshooting smart devices.

scholarly journals Recognizing Continuous and Discontinuous Adverse Drug Reaction Mentions from Social Media Using LSTM-CRF

2018 ◽  
Vol 2018 ◽  
pp. 1-8 ◽  
Author(s):  
Buzhou Tang ◽  
Jianglu Hu ◽  
Xiaolong Wang ◽  
Qingcai Chen
Keyword(s):  
Neural Networks ◽  
Social Media ◽  
Medical Information ◽  
Conditional Random Fields ◽  
Short Term Memory ◽  
Knowledge Bases ◽  
Short Term ◽  
External Knowledge ◽  
Long Short Term Memory ◽  
First Time

Social media in medicine, where patients can express their personal treatment experiences by personal computers and mobile devices, usually contains plenty of useful medical information, such as adverse drug reactions (ADRs); mining this useful medical information from social media has attracted more and more attention from researchers. In this study, we propose a deep neural network (called LSTM-CRF) combining long short-term memory (LSTM) neural networks (a type of recurrent neural networks) and conditional random fields (CRFs) to recognize ADR mentions from social media in medicine and investigate the effects of three factors on ADR mention recognition. The three factors are as follows: (1) representation for continuous and discontinuous ADR mentions: two novel representations, that is, “BIOHD” and “Multilabel,” are compared; (2) subject of posts: each post has a subject (i.e., drug here); and (3) external knowledge bases. Experiments conducted on a benchmark corpus, that is, CADEC, show that LSTM-CRF achieves better F-score than CRF; “Multilabel” is better in representing continuous and discontinuous ADR mentions than “BIOHD”; both subjects of comments and external knowledge bases are individually beneficial to ADR mention recognition. To the best of our knowledge, this is the first time to investigate deep neural networks to mine continuous and discontinuous ADRs from social media.

scholarly journals Robust-LSTM: A Novel Approach to Short-Traffic Flow Prediction Based on Signal Decomposition

2021 ◽  
Author(s):  
Erdem Doğan
Keyword(s):  
Traffic Flow ◽  
Short Term Memory ◽  
Performance Measurement System ◽  
Flow Data ◽  
Short Term ◽  
Term Memory ◽  
Traffic Flow Prediction ◽  
Flow Prediction ◽  
Novel Approach ◽  
Long Short Term Memory

Abstract Intelligent transport systems need accurate short-term traffic flow forecasts. However, developing a robust short-term traffic flow forecasting approach is a challenge due to the stochastic character of traffic flow. This study proposes a novel approach for short-term traffic flow prediction task namely Robust Long Short Term Memory (R-LSTM) based on Robust Empirical Mode Decomposing (REDM) algorithm and Long Short Term Memory (LSTM). Short-term traffic flow data provided from the Caltrans Performance Measurement System (PeMS) database were used in the training and testing of the model. The dataset was composed of traffic data collected by 25 traffic detectors on different freeways’ main lanes. The time resolution of the dataset was set to 15 minutes, and the Hampel preprocessing algorithm was applied for outlier elimination. The R-LSTM predictions were compared with the state-of-art models, utilizing RMSE, MSE, and MAPE as performance criteria. Performance analyzes for various periods show that R-LSTM is remarkably successful in all time periods. Moreover, developed model performance is significantly higher, especially during mid-day periods when traffic flow fluctuations are high. These results show that R-LSTM is a strong candidate for short-term traffic flow prediction, and can easily adapt to fluctuations in traffic flow. In addition, robust models for short-term predictions can be developed by applying the signal separation method to traffic flow data.

scholarly journals Where to Prune: Using LSTM to Guide End-to-end Pruning

2018 ◽  
Author(s):  
Jing Zhong ◽  
Guiguang Ding ◽  
Yuchen Guo ◽  
Jungong Han ◽  
Bin Wang
Keyword(s):  
Embedded System ◽  
Short Term Memory ◽  
Great Success ◽  
Layer By Layer ◽  
Computation Complexity ◽  
Short Term ◽  
Novel Approach ◽  
Comparable Accuracy ◽  
Model Size ◽  
Long Short Term Memory

Recent years have witnessed the great success of convolutional neural networks (CNNs) in many related fields. However, its huge model size and computation complexity bring in difficulty when deploying CNNs in some scenarios, like embedded system with low computation power. To address this issue, many works have been proposed to prune filters in CNNs to reduce computation. However, they mainly focus on seeking which filters are unimportant in a layer and then prune filters layer by layer or globally. In this paper, we argue that the pruning order is also very significant for model pruning. We propose a novel approach to figure out which layers should be pruned in each step.  First, we utilize a long short-term memory (LSTM) to learn the hierarchical characteristics of a network and generate a pruning decision for each layer, which is the main difference from previous works. Next, a channel-based method is adopted to evaluate the importance of filters in a to-be-pruned layer, followed by an accelerated recovery step. Experimental results demonstrate that our approach is capable of reducing 70.1% FLOPs for VGG and 47.5% for Resnet-56 with comparable accuracy. Also, the learning results seem to reveal the sensitivity of each network layer.

scholarly journals Convolutional Neural Networks with LSTM for Intrusion Detection

10.29007/j35r ◽  
2020 ◽  
Author(s):  
Mostofa Ahsan ◽  
Kendall Nygard
Keyword(s):  
Intrusion Detection ◽  
Hybrid Algorithm ◽  
Short Term Memory ◽  
High Accuracy ◽  
Short Term ◽  
Network Infrastructure ◽  
Network Intrusion ◽  
Novel Approach ◽  
Network Intrusions ◽  
Long Short Term Memory

A variety of attacks are regularly attempted at network infrastructure. With the increasing development of artificial intelligence algorithms, it has become effective to prevent network intrusion for more than two decades. Deep learning methods can achieve high accuracy with a low false alarm rate to detect network intrusions. A novel approach using a hybrid algorithm of Convolutional Neural Network (CNN) and Long Short Term Memory (LSTM) is introduced in this paper to provide improved intrusion detection. This bidirectional algorithm showed the highest known accuracy of 99.70% on a standard dataset known as NSL KDD. The performance of this algorithm is measured using precision, false positive, F1 score, and recall which found promising for deployment on live network infrastructure.

scholarly journals Conversation Modeling on Reddit Using a Graph-Structured LSTM

2018 ◽  
Vol 6 ◽  
pp. 121-132 ◽  
Author(s):  
Victoria Zayats ◽  
Mari Ostendorf
Keyword(s):  
Social Media ◽  
Short Term Memory ◽  
Short Term ◽  
Term Memory ◽  
Threaded Discussions ◽  
Novel Approach ◽  
Proposed Model ◽  
Long Short Term Memory ◽  
Bidirectional Lstm ◽  
Late Stages

This paper presents a novel approach for modeling threaded discussions on social media using a graph-structured bidirectional LSTM (long-short term memory) which represents both hierarchical and temporal conversation structure. In experiments with a task of predicting popularity of comments in Reddit discussions, the proposed model outperforms a node-independent architecture for different sets of input features. Analyses show a benefit to the model over the full course of the discussion, improving detection in both early and late stages. Further, the use of language cues with the bidirectional tree state updates helps with identifying controversial comments.

DLRCNeg: Deep Learning based Reading Comprehension by handling Negation

2019 ◽  
Author(s):  
Felicia Lilian J. ◽  
Sundarakantham K ◽  
Mercy Shalinie S.
Keyword(s):  
Reading Comprehension ◽  
Natural Language ◽  
Language Processing ◽  
Short Term Memory ◽  
Activation Function ◽  
Attention Mechanism ◽  
Question Type ◽  
Short Term ◽  
Novel Approach ◽  
Long Short Term Memory

Question Answer (QA) System for Reading Comprehension (RC) is a computerized approach to retrieve relevant response to the query posted by the users. The underlined concept in developing such a system is to build a human computer interaction. The interactions will be in natural language and we tend to use negation words as a part of our expressions. During the pre-processing stage in Natural Language Processing (NLP) task these negation words gets removed and hence the semantics gets changed. This remains to be an unsolved problem in QA system. In order to maintain the semantics we have proposed a novel approach Hybrid NLP based Bi-directional Long Short Term Memory (Bi-LSTM) with attention mechanism. It deals with the negation words and maintains the semantics of the sentence. We also focus on answering any factoid query (i.e. ’what’, ’when’, ’where’, ’who’) that is raised by the user. For this purpose, the use of attention mechanism with softmax activation function has obtained superior results that matches the question type and process the context information effectively. The experimental results are performed over the SQuAD dataset for reading comprehension and the Stanford Negation dataset is used to perform the negation in the RC sentence. The accuracy of the system over negation is obtained as 93.9% and over the QA system is 87%.

scholarly journals A Novel Approach for Detection of Fake News using Long Short Term Memory (LSTM)

2021 ◽  
Vol 10 (5) ◽  
pp. 3062-3066
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Text Mining ◽  
Short Term Memory ◽  
Online Media ◽  
Fake News ◽  
Short Term ◽  
Term Memory ◽  
Novel Approach ◽  
Long Short Term Memory

Online media for news consumption has doubtful advantages. From one perspective, it has minimal expense, simple access, and fast dispersal of data which leads individuals to search out and devour news from online media. On the other hand, it increases the wide spread of "counterfeit news", i.e., inferior quality news with purposefully bogus data. The broad spread of fake news contrarily affects people and society. Hence, fake news detection in social media has become an emerging research topic that is drawing attention from various researchers. In past, many creators proposed the utilization of text mining procedures and AI strategies to examine textual data and helps to foresee the believability of news. With more computational capacities and to deal with enormous datasets, deep learning models present a better presentation over customary text mining strategies and AI methods. Normally deep learning model, for example, LSTM model can identify complex patterns in the data. Long short term memory is a tree organized recurrent neural network (RNN) used to examine variable length sequential information. In our proposed framework we set up a fake news identification model dependent on LSTM neural network. Openly accessible unstructured news datasets are utilized to evaluate the exhibition of the model. The outcome shows the prevalence and exactness of LSTM model over the customary techniques specifically CNN for fake news recognition.

Sign in / Sign up

Export Citation Format

Share Document