scholarly journals Multi-Document Abstractive Summarization using Recursive Neural Network

2020 ◽  
Vol 9 (7) ◽  
pp. 364-370
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Text Summarization ◽  
Similarity Matrix ◽  
Text Documents ◽  
Short Text ◽  
Current State ◽  
Recursive Neural Networks ◽  
Semantic Concepts ◽  
Abstractive Summarization

Text summarization is an area of research with a goal to provide short text from huge text documents. Extractive text summarization methods have been extensively studied by many researchers. There are various type of multi document ranging from different formats to domains and topic specific. With the application of neural networks for text generation, interest for research in abstractive text summarization has increased significantly. This approach has been attempted for English and Telugu languages in this article. Recurrent neural networks are a subtype of recursive neural networks which try to predict the next sequence based on the current state and considering the information from previous states. The use of neural networks allows generation of summaries for long text sentences as well. The work implements semantic based filtering using a similarity matrix while keeping all stop-words. The similarity is calculated using semantic concepts and Jiang Similarity and making use of a Recurrent Neural Network (RNN) with an attention mechanism to generate summary. ROUGE score is used for measuring the performance of the applied method on Telugu and English langauges .

Unsupervised Multi-Document Abstractive Summarization Using Recursive Neural Network with Attention Mechanism

2020 ◽  
Vol 17 (9) ◽  
pp. 3867-3872
Author(s):  
Aniv Chakravarty ◽  
Jagadish S. Kallimani
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Attention Mechanism ◽  
Text Summarization ◽  
Text Generation ◽  
Text Documents ◽  
Current State ◽  
Semantic Concepts ◽  
Text Information ◽  
Abstractive Summarization

Text summarization is an active field of research with a goal to provide short and meaningful gists from large amount of text documents. Extractive text summarization methods have been extensively studied where text is extracted from the documents to build summaries. There are various type of multi document ranging from different formats to domains and topics. With the recent advancement in technology and use of neural networks for text generation, interest for research in abstractive text summarization has increased significantly. The use of graph based methods which handle semantic information has shown significant results. When given a set of documents of English text files, we make use of abstractive method and predicate argument structures to retrieve necessary text information and pass it through a neural network for text generation. Recurrent neural networks are a subtype of recursive neural networks which try to predict the next sequence based on the current state and considering the information from previous states. The use of neural networks allows generation of summaries for long text sentences as well. This paper implements a semantic based filtering approach using a similarity matrix while keeping all stop-words. The similarity is calculated using semantic concepts and Jiang–Conrath similarity and making use of a recurrent neural network with an attention mechanism to generate summary. ROUGE score is used for measuring accuracy, precision and recall scores.

scholarly journals A Pointer Generator Network Model to Automatic Text Summarization and Headline Generation

2019 ◽  
Vol 8 (5S3) ◽  
pp. 447-451
Keyword(s):  
Neural Network ◽  
Network Model ◽  
Recurrent Neural Network ◽  
Text Summarization ◽  
Daily Mail ◽  
Automatic Text Summarization ◽  
Generator Model ◽  
Abstractive Summarization ◽  
Automatic Text

In a world where information is growing rapidly every single day, we need tools to generate summary and headlines from text which is accurate as well as short and precise. In this paper, we have described a method for generating headlines from article. This is done by using hybrid pointer-generator network with attention distribution and coverage mechanism on article which generates abstractive summarization followed by the application of encoder-decoder recurrent neural network with LSTM unit to generate headlines from the summary. Hybrid pointer generator model helps in removing inaccuracy as well as repetitions. We have used CNN / Daily Mail as our dataset.

scholarly journals Language Semantics Interpretation with an Interaction-Based Recurrent Neural Network

2021 ◽  
Vol 3 (4) ◽  
pp. 922-945
Author(s):  
Shaw-Hwa Lo ◽  
Yiqiao Yin
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Language Processing ◽  
Text Classification ◽  
Search Algorithm ◽  
Greedy Search ◽  
Text Documents ◽  
Engineering Technique ◽  
Language Semantics ◽  
Sequential Models

Text classification is a fundamental language task in Natural Language Processing. A variety of sequential models are capable of making good predictions, yet there is a lack of connection between language semantics and prediction results. This paper proposes a novel influence score (I-score), a greedy search algorithm, called Backward Dropping Algorithm (BDA), and a novel feature engineering technique called the “dagger technique”. First, the paper proposes to use the novel influence score (I-score) to detect and search for the important language semantics in text documents that are useful for making good predictions in text classification tasks. Next, a greedy search algorithm, called the Backward Dropping Algorithm, is proposed to handle long-term dependencies in the dataset. Moreover, the paper proposes a novel engineering technique called the “dagger technique” that fully preserves the relationship between the explanatory variable and the response variable. The proposed techniques can be further generalized into any feed-forward Artificial Neural Networks (ANNs) and Convolutional Neural Networks (CNNs), and any neural network. A real-world application on the Internet Movie Database (IMDB) is used and the proposed methods are applied to improve prediction performance with an 81% error reduction compared to other popular peers if I-score and “dagger technique” are not implemented.

scholarly journals A HYBRID MODEL USING THE PRETRAINED BERT AND DEEP NEURAL NETWORKS WITH RICH FEATURE FOR EXTRACTIVE TEXT SUMMARIZATION

2021 ◽  
Vol 37 (2) ◽  
pp. 123-143
Author(s):  
Tuan Minh Luu ◽  
Huong Thanh Le ◽  
Tan Minh Hoang
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Neural Networks ◽  
Text Summarization ◽  
Training Dataset ◽  
Extractive Summarization ◽  
Input Text ◽  
Summarization System ◽  
Fully Connected

Deep neural networks have been applied successfully to extractive text summarization tasks with the accompany of large training datasets. However, when the training dataset is not large enough, these models reveal certain limitations that affect the quality of the system’s summary. In this paper, we propose an extractive summarization system basing on a Convolutional Neural Network and a Fully Connected network for sentence selection. The pretrained BERT multilingual model is used to generate embeddings vectors from the input text. These vectors are combined with TF-IDF values to produce the input of the text summarization system. Redundant sentences from the output summary are eliminated by the Maximal Marginal Relevance method. Our system is evaluated with both English and Vietnamese languages using CNN and Baomoi datasets, respectively. Experimental results show that our system achieves better results comparing to existing works using the same dataset. It confirms that our approach can be effectively applied to summarize both English and Vietnamese languages.

scholarly journals Incorporating Context-Relevant Knowledge into Convolutional Neural Networks for Short Text Classification

2019 ◽  
Vol 33 ◽  
pp. 10067-10068 ◽  
Author(s):  
Jingyun Xu ◽  
Yi Cai
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Convolutional Neural Network ◽  
Convolutional Neural Networks ◽  
Text Classification ◽  
Classification Methods ◽  
Short Text ◽  
Proposed Model ◽  
High Level ◽  
Context Features

Some text classification methods don’t work well on short texts due to the data sparsity. What’s more, they don’t fully exploit context-relevant knowledge. In order to tackle these problems, we propose a neural network to incorporate context-relevant knowledge into a convolutional neural network for short text classification. Our model consists of two modules. The first module utilizes two layers to extract concept and context features respectively and then employs an attention layer to extract those context-relevant concepts. The second module utilizes a convolutional neural network to extract high-level features from the word and the contextrelevant concept features. The experimental results on three datasets show that our proposed model outperforms the stateof-the-art models.

scholarly journals Template-Based Math Word Problem Solvers with Recursive Neural Networks

2019 ◽  
Vol 33 ◽  
pp. 7144-7151 ◽  
Author(s):  
Lei Wang ◽  
Dongxiang Zhang ◽  
Jipeng Zhang ◽  
Xing Xu ◽  
Lianli Gao ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Word Problem ◽  
Large Space ◽  
Wide Margin ◽  
Recursive Neural Networks ◽  
Recursive Neural Network ◽  
Operator Nodes ◽  
Problem Solvers ◽  
New Framework

The design of automatic solvers to arithmetic math word problems has attracted considerable attention in recent years and a large number of datasets and methods have been published. Among them, Math23K is the largest data corpus that is very helpful to evaluate the generality and robustness of a proposed solution. The best performer in Math23K is a seq2seq model based on LSTM to generate the math expression. However, the model suffers from performance degradation in large space of target expressions. In this paper, we propose a template-based solution based on recursive neural network for math expression construction. More specifically, we first apply a seq2seq model to predict a tree-structure template, with inferred numbers as leaf nodes and unknown operators as inner nodes. Then, we design a recursive neural network to encode the quantity with Bi-LSTM and self attention, and infer the unknown operator nodes in a bottom-up manner. The experimental results clearly establish the superiority of our new framework as we improve the accuracy by a wide margin in two of the largest datasets, i.e., from 58.1% to 66.9% in Math23K and from 62.8% to 66.8% in MAWPS.

OnSeS: A Novel Online Short Text Summarization Based on BM25 and Neural Network

2016 ◽  
Author(s):  
Jianwei Niu ◽  
Qingjuan Zhao ◽  
Lei Wang ◽  
Huan Chen ◽  
Mohammed Atiquzzaman ◽  
...  
Keyword(s):  
Neural Network ◽  
Text Summarization ◽  
Short Text

scholarly journals Testing the Reliability of Interpretable Neural Networks in Geoscience Using the Madden-Julian Oscillation

2020 ◽  
Author(s):  
Benjamin A. Toms ◽  
Karthik Kashinath ◽  
Da Yang ◽  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Test Case ◽  
Madden Julian Oscillation ◽  
State Variables ◽  
Current State ◽  
The Neural Network ◽  
Geophysical Phenomenon ◽  
The Neural Networks ◽  
Atmospheric State

Abstract. We test the reliability of two neural network interpretation techniques, backward optimization and layerwise relevance propagation, within geoscientific applications by applying them to a commonly studied geophysical phenomenon, the Madden-Julian Oscillation. The Madden-Julian Oscillation is a multi-scale pattern within the tropical atmosphere that has been extensively studied over the past decades, which makes it an ideal test case to ensure the interpretability methods can recover the current state of knowledge regarding its spatial structure. The neural networks can, indeed, reproduce the current state of knowledge and can also provide new insights into the seasonality of the Madden-Julian Oscillation and its relationships with atmospheric state variables. The neural network identifies the phase of the Madden-Julian Oscillation twice as accurately as linear regression, which means that nonlinearities used by the neural network are important to the structure of the Madden-Julian Oscillation. Interpretations of the neural network show that it accurately captures the spatial structures of the Madden-Julian Oscillation, suggest that the nonlinearities of the Madden-Julian Oscillation are manifested through the uniqueness of each event, and offer physically meaningful insights into its relationship with atmospheric state variables. We also use the interpretations to identify the seasonality of the MJO, and find that the conventionally defined extended seasons should be shifted later by one month. More generally, this study suggests that neural networks can be reliably interpreted for geoscientific applications and may thereby serve as a dependable method for testing geoscientific hypotheses.

scholarly journals Text Summary Generation Techniques

2020 ◽  
Vol 9 (7S) ◽  
pp. 50-54
Keyword(s):  
Pattern Recognition ◽  
Deep Learning ◽  
Empirical Evidence ◽  
Text Summarization ◽  
Text Documents ◽  
Observational Analysis ◽  
Abstractive Summarization

Pattern Recognition is pertinent field in autonomous text summarization for extraction of features from relative and non relative text documents. Here we provide empirical evidence that the method of Deep learning using RNN outperforms various techniques in terms of speed as well as metrics in abstractive summarization of multi-modal documents. We performed observational analysis on over 8 different techniques documented.

scholarly journals Predicting Customer Turnover Using Recursive Neural Networks

2021 ◽  
Vol 2021 ◽  
pp. 1-11
Author(s):  
Abdullah Jafari Chashmi ◽  
Vahid Rahmati ◽  
Behrouz Rezasoroush ◽  
Masumeh Motevalli Alamoti ◽  
Mohsen Askari ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Learning Algorithm ◽  
Customer Relationship ◽  
Loyalty Programs ◽  
Recurrent Nerve ◽  
The Neural Network ◽  
Recursive Neural Networks ◽  
The Neural Networks ◽  
A Company

The most valuable asset for a company is its customers’ base. As a result, customer relationship management (CRM) is an important task that drives companies. By identifying and understanding the valuable customer segments, appropriate marketing strategies can be used to enhance customer satisfaction and maintain loyalty, as well as increase company retention. Predicting customer turnover is an important tool for companies to stay competitive in a fast-growing market. In this paper, we use the recurrent nerve sketch to predict rejection based on the time series of the lifetime of the customer. In anticipation, a key aspect of identifying key triggers is to turn off. To overcome the weakness of recurrent neural networks, the research model of the combination of LRFMP with the neural network has been used. In this paper, it was found that clustering by LRFMP can be used to perform a more comprehensive analysis of customers’ turnover. In this solution, LRFMP is used to execute customer segregation. The objective is to provide a new framework for LRFMP for macrodata and macrodata analysis in order to increase the problem of business problem solving and customer depreciation. The results of the research show that the neural networks are capable of predicting the LRFMP precursors of the customers in an effective way. This model can be used in advocacy systems for advertising and loyalty programs management. In the previous research, the LRFM and RFM algorithms along with the neural network and the machine learning algorithm, etc., have been used, and in the proposed solution, the use of the LRFMP algorithm increases the accuracy of the desired.

Sign in / Sign up

Export Citation Format

Share Document