scholarly journals Extractive Text Summarization using Recurrent Neural Networks with Attention Mechanism

2021 ◽  
Author(s):  
Shimirwa Aline Valerie ◽  
Jian Xu
Keyword(s):  
Recurrent Neural Networks ◽  
State Of The Art ◽  
Attention Mechanism ◽  
Text Summarization ◽  
Data Driven ◽  
Future Research ◽  
Generalization Capability ◽  
Extractive Summarization ◽  
Benchmark Datasets ◽  
Set Up

Extractive summarization aims to select the most important sentences or words from a document to generate a summary. Traditional summarization approaches have relied extensively on features manually designed by humans. In this paper, based on the recurrent neural network equipped with the attention mechanism, we propose a data-driven technique. We set up a general framework that consists of a hierarchical sentence encoder and an attentionbased sentence extractor. The framework allows us to establish various extractive summarization models and explore them. Comprehensive experiments are conducted on two benchmark datasets, and experimental results show that training extractive models based on Reward Augmented Maximum Likelihood (RAML)can improve the model’s generalization capability. And we realize that complicated components of the state-of-the-art extractive models do not attain good performance over simpler ones. We hope that our work can give more hints for future research on extractive text summarization.

A Comprehensive Taxonomy of Dynamic Texture Representation

2023 ◽  
Vol 55 (1) ◽  
pp. 1-39
Author(s):  
Thanh Tuan Nguyen ◽  
Thanh Phuong Nguyen
Keyword(s):  
Large Scale ◽  
Environmental Changes ◽  
State Of The Art ◽  
The State ◽  
Future Research ◽  
Research Activities ◽  
Potential Applications ◽  
Benchmark Datasets ◽  
Negative Impacts ◽  
Made In

Representing dynamic textures (DTs) plays an important role in many real implementations in the computer vision community. Due to the turbulent and non-directional motions of DTs along with the negative impacts of different factors (e.g., environmental changes, noise, illumination, etc.), efficiently analyzing DTs has raised considerable challenges for the state-of-the-art approaches. For 20 years, many different techniques have been introduced to handle the above well-known issues for enhancing the performance. Those methods have shown valuable contributions, but the problems have been incompletely dealt with, particularly recognizing DTs on large-scale datasets. In this article, we present a comprehensive taxonomy of DT representation in order to purposefully give a thorough overview of the existing methods along with overall evaluations of their obtained performances. Accordingly, we arrange the methods into six canonical categories. Each of them is then taken in a brief presentation of its principal methodology stream and various related variants. The effectiveness levels of the state-of-the-art methods are then investigated and thoroughly discussed with respect to quantitative and qualitative evaluations in classifying DTs on benchmark datasets. Finally, we point out several potential applications and the remaining challenges that should be addressed in further directions. In comparison with two existing shallow DT surveys (i.e., the first one is out of date as it was made in 2005, while the newer one (published in 2016) is an inadequate overview), we believe that our proposed comprehensive taxonomy not only provides a better view of DT representation for the target readers but also stimulates future research activities.

SANTM: Efficient Self-attention-driven Network for Text Matching

2022 ◽  
Vol 22 (3) ◽  
pp. 1-21
Author(s):  
Prayag Tiwari ◽  
Amit Kumar Jaiswal ◽  
Sahil Garg ◽  
Ilsun You
Keyword(s):  
Natural Language ◽  
State Of The Art ◽  
The State ◽  
Attention Mechanism ◽  
Matching Problems ◽  
Attention Model ◽  
Extra Information ◽  
Textual Entailment ◽  
Benchmark Datasets ◽  
Text Matching

Self-attention mechanisms have recently been embraced for a broad range of text-matching applications. Self-attention model takes only one sentence as an input with no extra information, i.e., one can utilize the final hidden state or pooling. However, text-matching problems can be interpreted either in symmetrical or asymmetrical scopes. For instance, paraphrase detection is an asymmetrical task, while textual entailment classification and question-answer matching are considered asymmetrical tasks. In this article, we leverage attractive properties of self-attention mechanism and proposes an attention-based network that incorporates three key components for inter-sequence attention: global pointwise features, preceding attentive features, and contextual features while updating the rest of the components. Our model follows evaluation on two benchmark datasets cover tasks of textual entailment and question-answer matching. The proposed efficient Self-attention-driven Network for Text Matching outperforms the state of the art on the Stanford Natural Language Inference and WikiQA datasets with much fewer parameters.

Exploring the Role of Complexity, Content and Individual Differences in Aesthetic Reactions to Semi-Abstract Art Photographs

2020 ◽  
Vol 8 (1) ◽  
pp. 89-119
Author(s):  
Nathalie Vissers ◽  
Pieter Moors ◽  
Dominique Genin ◽  
Johan Wagemans
Keyword(s):  
Large Scale ◽  
A Priori ◽  
Data Driven ◽  
Future Research ◽  
Response Patterns ◽  
Online Study ◽  
Exploratory Data ◽  
Science Collaboration ◽  
Set Up ◽  
Different Response

Artistic photography is an interesting, but often overlooked, medium within the field of empirical aesthetics. Grounded in an art–science collaboration with art photographer Dominique Genin, this project focused on the relationship between the complexity of a photograph and its aesthetic appeal (beauty, pleasantness, interest). An artistic series of 24 semi-abstract photographs that play with multiple layers, recognisability vs unrecognizability and complexity was specifically created and selected for the project. A large-scale online study with a broad range of individuals (n = 453, varying in age, gender and art expertise) was set up. Exploratory data-driven analyses revealed two clusters of individuals, who responded differently to the photographs. Despite the semi-abstract nature of the photographs, differences seemed to be driven more consistently by the ‘content’ of the photograph than by its complexity levels. No consistent differences were found between clusters in age, gender or art expertise. Together, these results highlight the importance of exploratory, data-driven work in empirical aesthetics to complement and nuance findings from hypotheses-driven studies, as they allow to go further than a priori assumptions, to explore underlying clusters of participants with different response patterns, and to point towards new venues for future research. Data and code for the analyses reported in this article can be found at https://osf.io/2fws6/.

Deep Learning-based Roman-Urdu to Urdu Transliteration

2020 ◽  
pp. 2152001
Author(s):  
Mehreen Alam ◽  
Sibt ul Hussain
Keyword(s):  
Machine Translation ◽  
State Of The Art ◽  
Research Problem ◽  
Attention Mechanism ◽  
Data Driven ◽  
Neural Machine Translation ◽  
Parallel Corpus ◽  
Source Language ◽  
Data Driven Approach ◽  
Modern Machine

Attention-based encoder-decoder models have superseded conventional techniques due to their unmatched performance on many neural machine translation problems. Usually, the encoders and decoders are two recurrent neural networks where the decoder is directed to focus on relevant parts of the source language using attention mechanism. This data-driven approach leads to generic and scalable solutions with no reliance on manual hand-crafted features. To the best of our knowledge, none of the modern machine translation approaches has been applied to address the research problem of Urdu machine transliteration. Ours is the first attempt to apply the deep neural network-based encoder-decoder using attention mechanism to address the aforementioned problem using Roman-Urdu and Urdu parallel corpus. To this end, we present (i) the first ever Roman-Urdu to Urdu parallel corpus of 1.1 million sentences, (ii) three state of the art encoder-decoder models, and (iii) a detailed empirical analysis of these three models on the Roman-Urdu to Urdu parallel corpus. Overall, attention-based model gives state-of-the-art performance with the benchmark of 70 BLEU score. Our qualitative experimental evaluation shows that our models generate coherent transliterations which are grammatically and logically correct.

scholarly journals A Span-based Joint Model for Opinion Target Extraction and Target Sentiment Classification

2019 ◽  
Author(s):  
Yan Zhou ◽  
Longtao Huang ◽  
Tao Guo ◽  
Jizhong Han ◽  
Songlin Hu
Keyword(s):  
Sentiment Analysis ◽  
State Of The Art ◽  
Joint Model ◽  
The State ◽  
Attention Mechanism ◽  
Sentiment Classification ◽  
Global Information ◽  
Target Extraction ◽  
Benchmark Datasets ◽  
Tagging Methods

Target-Based Sentiment Analysis aims at extracting opinion targets and classifying the sentiment polarities expressed on each target. Recently, token based sequence tagging methods have been successfully applied to jointly solve the two tasks, which aims to predict a tag for each token. Since they do not treat a target containing several words as a whole, it might be difficult to make use of the global information to identify that opinion target, leading to incorrect extraction. Independently predicting the sentiment for each token may also lead to sentiment inconsistency for different words in an opinion target. In this paper, inspired by span-based methods in NLP, we propose a simple and effective joint model to conduct extraction and classification at span level rather than token level. Our model first emulates spans with one or more tokens and learns their representation based on the tokens inside. And then, a span-aware attention mechanism is designed to compute the sentiment information towards each span. Extensive experiments on three benchmark datasets show that our model consistently outperforms the state-of-the-art methods.

scholarly journals Text Summarization using Extractive and Abstractive Methods

2021 ◽  
Vol 40 ◽  
pp. 03023
Author(s):  
Saurabh Varade ◽  
Ejaaz Sayyed ◽  
Vaibhavi Nagtode ◽  
Shilpa Shinde
Keyword(s):  
Input Sequence ◽  
Link Analysis ◽  
Attention Mechanism ◽  
Text Summarization ◽  
Ranking Algorithm ◽  
Text File ◽  
Extractive Summarization ◽  
Ranking Algorithms ◽  
Original Meaning ◽  
Abstractive Summarization

Text Summarization is a process where a huge text file is converted into summarized version which will preserve the original meaning and context. The main aim of any text summarization is to provide a accurate and precise summary. One approach is to use a sentence ranking algorithm. This comes under extractive summarization. Here, a graph based ranking algorithm is used to rank the sentences in the text and then top k-scored sentences are included in the summary. The most widely used algorithm to decide the importance of any vertex in a graph based on the information retrieved from the graph is Graph Based Ranking Algorithm. TextRank is one of the most efficient ranking algorithms which is used for Web link analysis that is for measuring the importance of website pages. Another approach is abstractive summarization where a LSTM encoder decoder model is used along with attention mechanism which focuses on some important words from the input. Encoder encodes the input sequence and decoder along with attention mechanism gives the summary as the output.

scholarly journals DUAL TRANSFORMER ENCODERS FOR SESSION-BASED RECOMMENDATION

2021 ◽  
Vol 37 (4) ◽  
pp. 511-527
Author(s):  
Pham Hoang Anh ◽  
Ngo Xuan Bach ◽  
Tu Minh Phuong
Keyword(s):  
State Of The Art ◽  
Attention Mechanism ◽  
Sequential Patterns ◽  
Evaluation Metrics ◽  
Main Interest ◽  
Recent Advances ◽  
Benchmark Datasets

When long-term user proles are not available, session-based recommendation methods are used to predict the user's next actions from anonymous sessions-based data. Recent advances in session-based recommendation highlight the necessity of modeling not only user sequential behaviors but also the user's main interest in a session, while avoiding the eect of unintended clicks causing interest drift of the user. In this work, we propose a Dual Transformer Encoder Recommendation model (DTER) as a solution to address this requirement. The idea is to combine the following recipes: (1) a Transformer-based model with dual encoders capable of modeling both sequential patterns and the main interest of the user in a session; (2) a new recommendation model that is designed for learning richer session contexts by conditioning on all permutations of the session prex. This approach provides a unied framework for leveraging the ability of the Transformer's self-attention mechanism in modeling session sequences while taking into account the user's main interest in the session. We empirically evaluate the proposed method on two benchmark datasets. The results show that DTER outperforms state-of-the-art session-based recommendation methods on common evaluation metrics.

Data-driven Crowd Modeling Techniques: A Survey

2022 ◽  
Vol 32 (1) ◽  
pp. 1-33
Author(s):  
Jinghui Zhong ◽  
Dongrui Li ◽  
Zhixing Huang ◽  
Chengyu Lu ◽  
Wentong Cai
Keyword(s):  
Game Design ◽  
State Of The Art ◽  
Crowd Simulation ◽  
Data Driven ◽  
Future Research ◽  
Crowd Modeling ◽  
The Past ◽  
Modeling Techniques ◽  
Comprehensive Survey ◽  
Data Driven Modeling

Data-driven crowd modeling has now become a popular and effective approach for generating realistic crowd simulation and has been applied to a range of applications, such as anomaly detection and game design. In the past decades, a number of data-driven crowd modeling techniques have been proposed, providing many options for people to generate virtual crowd simulation. This article provides a comprehensive survey of these state-of-the-art data-driven modeling techniques. We first describe the commonly used datasets for crowd modeling. Then, we categorize and discuss the state-of-the-art data-driven crowd modeling methods. After that, data-driven crowd model validation techniques are discussed. Finally, six promising future research topics of data-driven crowd modeling are discussed.

scholarly journals Attention as Relation: Learning Supervised Multi-head Self-Attention for Relation Extraction

2020 ◽  
Author(s):  
Jie Liu ◽  
Shaowei Chen ◽  
Bingquan Wang ◽  
Jiaxin Zhang ◽  
Na Li ◽  
...  
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
State Of The Art ◽  
Relation Extraction ◽  
Attention Mechanism ◽  
Entity Extraction ◽  
Relation Type ◽  
Benchmark Datasets ◽  
Relation Learning

Joint entity and relation extraction is critical for many natural language processing (NLP) tasks, which has attracted increasing research interest. However, it is still faced with the challenges of identifying the overlapping relation triplets along with the entire entity boundary and detecting the multi-type relations. In this paper, we propose an attention-based joint model, which mainly contains an entity extraction module and a relation detection module, to address the challenges. The key of our model is devising a supervised multi-head self-attention mechanism as the relation detection module to learn the token-level correlation for each relation type separately. With the attention mechanism, our model can effectively identify overlapping relations and flexibly predict the relation type with its corresponding intensity. To verify the effectiveness of our model, we conduct comprehensive experiments on two benchmark datasets. The experimental results demonstrate that our model achieves state-of-the-art performances.

The State-of-the-Art in Structural Integrity Management: A Review and Proposed Data-Driven Approach

2019 ◽  
Author(s):  
YeongAe Heo
Keyword(s):  
Risk Management ◽  
Environmental Conditions ◽  
Structural Integrity ◽  
State Of The Art ◽  
Cost Effective ◽  
Offshore Structures ◽  
Data Driven ◽  
Future Research ◽  
Network Systems ◽  
Management Approaches

Abstract Probabilistic risk-based approaches have been used for cost-effective structural design and maintenance guidelines. The effectiveness of these provisions, however, has yet to be adequately validated. Also, current risk management approaches are hardly adaptable to changes in operational and environmental conditions as well as structural properties over the service life of structures. As the need and demand of real-time monitoring systems have increased dramatically for high-value and high-risk facilities such as offshore structures particularly, it is important to discuss directions for future research to advance the risk-based management approaches by utilizing the invaluable “big-scale” field data obtained from sensor network systems. Therefore, knowledge gaps in the current state-of-the-art structural risk management approaches are discussed in this paper. Subsequently, a novel risk management framework is presented in this paper integrating physics-based data into a data-driven decision model. The proposed decision framework will improve system adaptability to future performance requirements due to changing operational and environmental conditions and will be applicable to any structural systems instrumented by sophisticated SHM systems such as complex naval and marine systems.

Sign in / Sign up

Export Citation Format

Share Document