Improving the readability and saliency of abstractive text summarization using combination of deep neural networks equipped with auxiliary attention mechanism

Deep neural networks have been applied successfully to extractive text summarization tasks with the accompany of large training datasets. However, when the training dataset is not large enough, these models reveal certain limitations that affect the quality of the system’s summary. In this paper, we propose an extractive summarization system basing on a Convolutional Neural Network and a Fully Connected network for sentence selection. The pretrained BERT multilingual model is used to generate embeddings vectors from the input text. These vectors are combined with TF-IDF values to produce the input of the text summarization system. Redundant sentences from the output summary are eliminated by the Maximal Marginal Relevance method. Our system is evaluated with both English and Vietnamese languages using CNN and Baomoi datasets, respectively. Experimental results show that our system achieves better results comparing to existing works using the same dataset. It confirms that our approach can be effectively applied to summarize both English and Vietnamese languages.

Download Full-text

Unsupervised Multi-Document Abstractive Summarization Using Recursive Neural Network with Attention Mechanism

Journal of Computational and Theoretical Nanoscience ◽

10.1166/jctn.2020.8976 ◽

2020 ◽

Vol 17 (9) ◽

pp. 3867-3872

Author(s):

Aniv Chakravarty ◽

Jagadish S. Kallimani

Keyword(s):

Neural Network ◽

Neural Networks ◽

Attention Mechanism ◽

Text Summarization ◽

Text Generation ◽

Text Documents ◽

Current State ◽

Semantic Concepts ◽

Text Information ◽

Abstractive Summarization

Text summarization is an active field of research with a goal to provide short and meaningful gists from large amount of text documents. Extractive text summarization methods have been extensively studied where text is extracted from the documents to build summaries. There are various type of multi document ranging from different formats to domains and topics. With the recent advancement in technology and use of neural networks for text generation, interest for research in abstractive text summarization has increased significantly. The use of graph based methods which handle semantic information has shown significant results. When given a set of documents of English text files, we make use of abstractive method and predicate argument structures to retrieve necessary text information and pass it through a neural network for text generation. Recurrent neural networks are a subtype of recursive neural networks which try to predict the next sequence based on the current state and considering the information from previous states. The use of neural networks allows generation of summaries for long text sentences as well. This paper implements a semantic based filtering approach using a similarity matrix while keeping all stop-words. The similarity is calculated using semantic concepts and Jiang–Conrath similarity and making use of a recurrent neural network with an attention mechanism to generate summary. ROUGE score is used for measuring accuracy, precision and recall scores.

Download Full-text

Hierarchical Human-Like Deep Neural Networks for Abstractive Text Summarization

IEEE Transactions on Neural Networks and Learning Systems ◽

10.1109/tnnls.2020.3008037 ◽

2020 ◽

pp. 1-14

Author(s):

Min Yang ◽

Chengming Li ◽

Ying Shen ◽

Qingyao Wu ◽

Zhou Zhao ◽

...

Keyword(s):

Neural Networks ◽

Deep Neural Networks ◽

Text Summarization

Download Full-text

Learning a Hierarchical Global Attention for Image Classification

Future Internet ◽

10.3390/fi12110178 ◽

2020 ◽

Vol 12 (11) ◽

pp. 178

Author(s):

Kerang Cao ◽

Jingyu Gao ◽

Kwang-nam Choi ◽

Lini Duan

Keyword(s):

Neural Networks ◽

Image Classification ◽

Deep Neural Networks ◽

Matrix Multiplication ◽

Attention Mechanism ◽

Feature Representation ◽

Global Information ◽

Computation Complexity ◽

Computational Costs ◽

Large Effort

To classify the image material on the internet, the deep learning methodology, especially deep neural network, is the most optimal and costliest method of all computer vision methods. Convolutional neural networks (CNNs) learn a comprehensive feature representation by exploiting local information with a fixed receptive field, demonstrating distinguished capacities on image classification. Recent works concentrate on efficient feature exploration, which neglect the global information for holistic consideration. There is large effort to reduce the computational costs of deep neural networks. Here, we provide a hierarchical global attention mechanism that improve the network representation with restricted increase of computation complexity. Different from nonlocal-based methods, the hierarchical global attention mechanism requires no matrix multiplication and can be flexibly applied in various modern network designs. Experimental results demonstrate that proposed hierarchical global attention mechanism can conspicuously improve the image classification precision—a reduction of 7.94% and 16.63% percent in Top 1 and Top 5 errors separately—with little increase of computation complexity (6.23%) in comparison to competing approaches.

Download Full-text