scholarly journals Abstractive Arabic Text Summarization Based on Deep Learning

2022 ◽  
Vol 2022 ◽  
pp. 1-14
Author(s):  
Y.M. Wazery ◽  
Marwa E. Saleh ◽  
Abdullah Alharbi ◽  
Abdelmgeid A. Ali
Keyword(s):  
Short Term Memory ◽  
Attention Mechanism ◽  
Text Summarization ◽  
Arabic Text ◽  
Short Term ◽  
Term Memory ◽  
Long Short Term Memory ◽  
Arabic Text Summarization ◽  
Abstractive Summarization ◽  
Research Studies

Text summarization (TS) is considered one of the most difficult tasks in natural language processing (NLP). It is one of the most important challenges that stand against the modern computer system’s capabilities with all its new improvement. Many papers and research studies address this task in literature but are being carried out in extractive summarization, and few of them are being carried out in abstractive summarization, especially in the Arabic language due to its complexity. In this paper, an abstractive Arabic text summarization system is proposed, based on a sequence-to-sequence model. This model works through two components, encoder and decoder. Our aim is to develop the sequence-to-sequence model using several deep artificial neural networks to investigate which of them achieves the best performance. Different layers of Gated Recurrent Units (GRU), Long Short-Term Memory (LSTM), and Bidirectional Long Short-Term Memory (BiLSTM) have been used to develop the encoder and the decoder. In addition, the global attention mechanism has been used because it provides better results than the local attention mechanism. Furthermore, AraBERT preprocess has been applied in the data preprocessing stage that helps the model to understand the Arabic words and achieves state-of-the-art results. Moreover, a comparison between the skip-gram and the continuous bag of words (CBOW) word2Vec word embedding models has been made. We have built these models using the Keras library and run-on Google Colab Jupiter notebook to run seamlessly. Finally, the proposed system is evaluated through ROUGE-1, ROUGE-2, ROUGE-L, and BLEU evaluation metrics. The experimental results show that three layers of BiLSTM hidden states at the encoder achieve the best performance. In addition, our proposed system outperforms the other latest research studies. Also, the results show that abstractive summarization models that use the skip-gram word2Vec model outperform the models that use the CBOW word2Vec model.

scholarly journals Dam Deformation Interpretation and Prediction Based on a Long Short-Term Memory Model Coupled with an Attention Mechanism

2021 ◽  
Vol 11 (14) ◽  
pp. 6625
Author(s):  
Yan Su ◽  
Kailiang Weng ◽  
Chuan Lin ◽  
Zeqin Chen
Keyword(s):  
Short Term Memory ◽  
Attention Mechanism ◽  
Impact Factors ◽  
Nonlinear Prediction ◽  
Time Dimension ◽  
Short Term ◽  
Deformation Prediction ◽  
Term Memory ◽  
Long Short Term Memory ◽  
Dam Deformation

An accurate dam deformation prediction model is vital to a dam safety monitoring system, as it helps assess and manage dam risks. Most traditional dam deformation prediction algorithms ignore the interpretation and evaluation of variables and lack qualitative measures. This paper proposes a data processing framework that uses a long short-term memory (LSTM) model coupled with an attention mechanism to predict the deformation response of a dam structure. First, the random forest (RF) model is introduced to assess the relative importance of impact factors and screen input variables. Secondly, the density-based spatial clustering of applications with noise (DBSCAN) method is used to identify and filter the equipment based abnormal values to reduce the random error in the measurements. Finally, the coupled model is used to focus on important factors in the time dimension in order to obtain more accurate nonlinear prediction results. The results of the case study show that, of all tested methods, the proposed coupled method performed best. In addition, it was found that temperature and water level both have significant impacts on dam deformation and can serve as reliable metrics for dam management.

A Method Based on Attention Mechanism using Bidirectional Long-Short Term Memory(BLSTM) for Question Answering

2021 ◽  
Author(s):  
Seyed Vahid Moravvej ◽  
Mohammad Javad Maleki Kahaki ◽  
Moein Salimi Sartakhti ◽  
Abdolreza Mirzaei
Keyword(s):  
Question Answering ◽  
Short Term Memory ◽  
Attention Mechanism ◽  
Short Term ◽  
Term Memory ◽  
Long Short Term Memory

scholarly journals An Optimized Abstractive Text Summarization Model Using Peephole Convolutional LSTM

Symmetry ◽  
2019 ◽  
Vol 11 (10) ◽  
pp. 1290 ◽  
Author(s):  
Rahman ◽  
Siddiqui
Keyword(s):  
Language Processing ◽  
Short Term Memory ◽  
State Of The Art ◽  
Text Summarization ◽  
Short Term ◽  
Term Memory ◽  
Semantic Coherence ◽  
Long Short Term Memory ◽  
Central Composite ◽  
Convolutional Lstm

Abstractive text summarization that generates a summary by paraphrasing a long text remains an open significant problem for natural language processing. In this paper, we present an abstractive text summarization model, multi-layered attentional peephole convolutional LSTM (long short-term memory) (MAPCoL) that automatically generates a summary from a long text. We optimize parameters of MAPCoL using central composite design (CCD) in combination with the response surface methodology (RSM), which gives the highest accuracy in terms of summary generation. We record the accuracy of our model (MAPCoL) on a CNN/DailyMail dataset. We perform a comparative analysis of the accuracy of MAPCoL with that of the state-of-the-art models in different experimental settings. The MAPCoL also outperforms the traditional LSTM-based models in respect of semantic coherence in the output summary.

scholarly journals An LSTM-Based Method with Attention Mechanism for Travel Time Prediction

Sensors ◽  
2019 ◽  
Vol 19 (4) ◽  
pp. 861 ◽  
Author(s):  
Xiangdong Ran ◽  
Zhiguang Shan ◽  
Yufei Fang ◽  
Chuang Lin
Keyword(s):  
Short Term Memory ◽  
Attention Mechanism ◽  
Traffic Prediction ◽  
Travel Time Prediction ◽  
Short Term ◽  
Term Memory ◽  
Proposed Model ◽  
Departure Time ◽  
Long Short Term Memory

Traffic prediction is based on modeling the complex non-linear spatiotemporal traffic dynamics in road network. In recent years, Long Short-Term Memory has been applied to traffic prediction, achieving better performance. The existing Long Short-Term Memory methods for traffic prediction have two drawbacks: they do not use the departure time through the links for traffic prediction, and the way of modeling long-term dependence in time series is not direct in terms of traffic prediction. Attention mechanism is implemented by constructing a neural network according to its task and has recently demonstrated success in a wide range of tasks. In this paper, we propose an Long Short-Term Memory-based method with attention mechanism for travel time prediction. We present the proposed model in a tree structure. The proposed model substitutes a tree structure with attention mechanism for the unfold way of standard Long Short-Term Memory to construct the depth of Long Short-Term Memory and modeling long-term dependence. The attention mechanism is over the output layer of each Long Short-Term Memory unit. The departure time is used as the aspect of the attention mechanism and the attention mechanism integrates departure time into the proposed model. We use AdaGrad method for training the proposed model. Based on the datasets provided by Highways England, the experimental results show that the proposed model can achieve better accuracy than the Long Short-Term Memory and other baseline methods. The case study suggests that the departure time is effectively employed by using attention mechanism.

Multi‐dimensional long short‐term memory networks for artificial Arabic text recognition in news video

2018 ◽  
Vol 12 (5) ◽  
pp. 710-719 ◽  
Author(s):  
Oussama Zayene ◽  
Sameh Masmoudi Touj ◽  
Jean Hennebert ◽  
Rolf Ingold ◽  
Najoua Essoukri Ben Amara
Keyword(s):  
Short Term Memory ◽  
Text Recognition ◽  
Arabic Text ◽  
Short Term ◽  
Term Memory ◽  
News Video ◽  
Long Short Term Memory

scholarly journals Convolutional Auto-encoding of Sentence Topics for Image Paragraph Generation

2019 ◽  
Author(s):  
Jing Wang ◽  
Yingwei Pan ◽  
Ting Yao ◽  
Jinhui Tang ◽  
Tao Mei
Keyword(s):  
Coherent Structure ◽  
Topic Modeling ◽  
Short Term Memory ◽  
State Of The Art ◽  
Attention Mechanism ◽  
Visual Content ◽  
Short Term ◽  
Term Memory ◽  
Sentence Level ◽  
Long Short Term Memory

Image paragraph generation is the task of producing a coherent story (usually a paragraph) that describes the visual content of an image. The problem nevertheless is not trivial especially when there are multiple descriptive and diverse gists to be considered for paragraph generation, which often happens in real images. A valid question is how to encapsulate such gists/topics that are worthy of mention from an image, and then describe the image from one topic to another but holistically with a coherent structure. In this paper, we present a new design --- Convolutional Auto-Encoding (CAE) that purely employs convolutional and deconvolutional auto-encoding framework for topic modeling on the region-level features of an image. Furthermore, we propose an architecture, namely CAE plus Long Short-Term Memory (dubbed as CAE-LSTM), that novelly integrates the learnt topics in support of paragraph generation. Technically, CAE-LSTM capitalizes on a two-level LSTM-based paragraph generation framework with attention mechanism. The paragraph-level LSTM captures the inter-sentence dependency in a paragraph, while sentence-level LSTM is to generate one sentence which is conditioned on each learnt topic. Extensive experiments are conducted on Stanford image paragraph dataset, and superior results are reported when comparing to state-of-the-art approaches. More remarkably, CAE-LSTM increases CIDEr performance from 20.93% to 25.15%.

Sign in / Sign up

Export Citation Format

Share Document