scholarly journals Abstract Text Summarization with a Convolutional Seq2seq Model

2019 ◽  
Vol 9 (8) ◽  
pp. 1665 ◽  
Author(s):  
Yong Zhang ◽  
Dan Li ◽  
Yuheng Wang ◽  
Yang Fang ◽  
Weidong Xiao
Keyword(s):  
State Of The Art ◽  
Real Life ◽  
Generative Model ◽  
Attention Mechanism ◽  
Text Summarization ◽  
Model Based ◽  
Main Ideas

Abstract text summarization aims to offer a highly condensed and valuable information that expresses the main ideas of the text. Most previous researches focus on extractive models. In this work, we put forward a new generative model based on convolutional seq2seq architecture. A hierarchical CNN framework is much more efficient than the conventional RNN seq2seq models. We also equip our model with a copying mechanism to deal with the rare or unseen words. Additionally, we incorporate a hierarchical attention mechanism to model the keywords and key sentences simultaneously. Finally we verify our model on two real-life datasets, GigaWord and DUC corpus. The experiment results verify the effectiveness of our model as it outperforms state-of-the-art alternatives consistently and statistical significantly.

scholarly journals Extractive Text Summarization using Recurrent Neural Networks with Attention Mechanism

2021 ◽  
Author(s):  
Shimirwa Aline Valerie ◽  
Jian Xu
Keyword(s):  
Recurrent Neural Networks ◽  
State Of The Art ◽  
Attention Mechanism ◽  
Text Summarization ◽  
Data Driven ◽  
Future Research ◽  
Generalization Capability ◽  
Extractive Summarization ◽  
Benchmark Datasets ◽  
Set Up

Extractive summarization aims to select the most important sentences or words from a document to generate a summary. Traditional summarization approaches have relied extensively on features manually designed by humans. In this paper, based on the recurrent neural network equipped with the attention mechanism, we propose a data-driven technique. We set up a general framework that consists of a hierarchical sentence encoder and an attentionbased sentence extractor. The framework allows us to establish various extractive summarization models and explore them. Comprehensive experiments are conducted on two benchmark datasets, and experimental results show that training extractive models based on Reward Augmented Maximum Likelihood (RAML)can improve the model’s generalization capability. And we realize that complicated components of the state-of-the-art extractive models do not attain good performance over simpler ones. We hope that our work can give more hints for future research on extractive text summarization.

AutoLinker: Automatic Fragment Linking with Deep Conditional Transformer Neural Networks

2020 ◽  
Author(s):  
Yuyao Yang ◽  
Shuangjia Zheng ◽  
Shimin Su ◽  
Jun Xu ◽  
Hongming Chen
Keyword(s):  
Neural Networks ◽  
Drug Discovery ◽  
Drug Design ◽  
State Of The Art ◽  
The State ◽  
Generative Model ◽  
Lead Optimization ◽  
Scaffold Hopping ◽  
Reference Models ◽  
Lead Generation

Fragment based drug design represents a promising drug discovery paradigm complimentary to the traditional HTS based lead generation strategy. How to link fragment structures to increase compound affinity is remaining a challenge task in this paradigm. Hereby a novel deep generative model (AutoLinker) for linking fragments is developed with the potential for applying in the fragment-based lead generation scenario. The state-of-the-art transformer architecture was employed to learn the linker grammar and generate novel linker. Our results show that, given starting fragments and user customized linker constraints, our AutoLinker model can design abundant drug-like molecules fulfilling these constraints and its performance was superior to other reference models. Moreover, several examples were showcased that AutoLinker can be useful tools for carrying out drug design tasks such as fragment linking, lead optimization and scaffold hopping.

Lattice Boltzmann for Turbulence Modeling

2018 ◽  
Author(s):  
Sauro Succi
Keyword(s):  
Turbulent Flows ◽  
Lattice Boltzmann ◽  
Turbulence Modeling ◽  
Real Life ◽  
Practical Interest ◽  
Reynolds Numbers ◽  
Full Resolution ◽  
To Come ◽  
Highly Turbulent Flows ◽  
Main Ideas

This chapter introduces the main ideas behind the application of LBE methods to the problem of turbulence modeling, namely the simulation of flows which contain scales of motion too small to be resolved on present-day and foreseeable future computers. Many real-life flows of practical interest exhibit Reynolds numbers far too high to be directly simulated in full resolution on present-day computers and arguably for many years to come. This raises the challenge of predicting the behavior of highly turbulent flows without directly simulating all scales of motion which take part to turbulence dynamics, but only those that fall within the computer resolution at hand.

An interpretable mechanism for personalized recommendation based on cross feature

2021 ◽  
pp. 1-12
Author(s):  
Lv YE ◽  
Yue Yang ◽  
Jian-Xu Zeng
Keyword(s):  
Recommender System ◽  
Online Shopping ◽  
Side Information ◽  
Attention Mechanism ◽  
Personalized Recommendation ◽  
Model Based ◽  
Embedded Model ◽  
Feature Attention ◽  
Match Score

The existing recommender system provides personalized recommendation service for users in online shopping, entertainment, and other activities. In order to improve the probability of users accepting the system’s recommendation service, compared with the traditional recommender system, the interpretable recommender system will give the recommendation reasons and results at the same time. In this paper, an interpretable recommendation model based on XGBoost tree is proposed to obtain comprehensible and effective cross features from side information. The results are input into the embedded model based on attention mechanism to capture the invisible interaction among user IDs, item IDs and cross features. The captured interactions are used to predict the match score between the user and the recommended item. Cross-feature attention score is used to generate different recommendation reasons for different user-items.Experimental results show that the proposed algorithm can guarantee the quality of recommendation. The transparency and readability of the recommendation process has been improved by providing reference reasons. This method can help users better understand the recommendation behavior of the system and has certain enlightenment to help the recommender system become more personalized and intelligent.

scholarly journals Improved SinGAN Integrated with an Attentional Mechanism for Remote Sensing Image Classification

2021 ◽  
Vol 13 (9) ◽  
pp. 1713
Author(s):  
Songwei Gu ◽  
Rui Zhang ◽  
Hongxia Luo ◽  
Mengyao Li ◽  
Huamei Feng ◽  
...  
Keyword(s):  
Remote Sensing ◽  
Real Life ◽  
Attention Mechanism ◽  
Training Data ◽  
Generative Adversarial Networks ◽  
Natural Image ◽  
Remote Sensing Images ◽  
Training Time ◽  
Adversarial Networks ◽  
Remote Sensing Image Classification

Deep learning is an important research method in the remote sensing field. However, samples of remote sensing images are relatively few in real life, and those with markers are scarce. Many neural networks represented by Generative Adversarial Networks (GANs) can learn from real samples to generate pseudosamples, rather than traditional methods that often require more time and man-power to obtain samples. However, the generated pseudosamples often have poor realism and cannot be reliably used as the basis for various analyses and applications in the field of remote sensing. To address the abovementioned problems, a pseudolabeled sample generation method is proposed in this work and applied to scene classification of remote sensing images. The improved unconditional generative model that can be learned from a single natural image (Improved SinGAN) with an attention mechanism can effectively generate enough pseudolabeled samples from a single remote sensing scene image sample. Pseudosamples generated by the improved SinGAN model have stronger realism and relatively less training time, and the extracted features are easily recognized in the classification network. The improved SinGAN can better identify sub-jects from images with complex ground scenes compared with the original network. This mechanism solves the problem of geographic errors of generated pseudosamples. This study incorporated the generated pseudosamples into training data for the classification experiment. The result showed that the SinGAN model with the integration of the attention mechanism can better guarantee feature extraction of the training data. Thus, the quality of the generated samples is improved and the classification accuracy and stability of the classification network are also enhanced.

scholarly journals Head-to-head comparison of clustering methods for heterogeneous data: a simulation-driven benchmark

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Gregoire Preud’homme ◽  
Kevin Duarte ◽  
Kevin Dalleau ◽  
Claire Lacomblez ◽  
Emmanuel Bresso ◽  
...  
Keyword(s):  
Hierarchical Clustering ◽  
Latent Class ◽  
Latent Class Model ◽  
Real Life ◽  
Heterogeneous Data ◽  
Mixed Data ◽  
Categorical Variables ◽  
Clustering Methods ◽  
Model Based ◽  
Partitioning Around Medoids

AbstractThe choice of the most appropriate unsupervised machine-learning method for “heterogeneous” or “mixed” data, i.e. with both continuous and categorical variables, can be challenging. Our aim was to examine the performance of various clustering strategies for mixed data using both simulated and real-life data. We conducted a benchmark analysis of “ready-to-use” tools in R comparing 4 model-based (Kamila algorithm, Latent Class Analysis, Latent Class Model [LCM] and Clustering by Mixture Modeling) and 5 distance/dissimilarity-based (Gower distance or Unsupervised Extra Trees dissimilarity followed by hierarchical clustering or Partitioning Around Medoids, K-prototypes) clustering methods. Clustering performances were assessed by Adjusted Rand Index (ARI) on 1000 generated virtual populations consisting of mixed variables using 7 scenarios with varying population sizes, number of clusters, number of continuous and categorical variables, proportions of relevant (non-noisy) variables and degree of variable relevance (low, mild, high). Clustering methods were then applied on the EPHESUS randomized clinical trial data (a heart failure trial evaluating the effect of eplerenone) allowing to illustrate the differences between different clustering techniques. The simulations revealed the dominance of K-prototypes, Kamila and LCM models over all other methods. Overall, methods using dissimilarity matrices in classical algorithms such as Partitioning Around Medoids and Hierarchical Clustering had a lower ARI compared to model-based methods in all scenarios. When applying clustering methods to a real-life clinical dataset, LCM showed promising results with regard to differences in (1) clinical profiles across clusters, (2) prognostic performance (highest C-index) and (3) identification of patient subgroups with substantial treatment benefit. The present findings suggest key differences in clustering performance between the tested algorithms (limited to tools readily available in R). In most of the tested scenarios, model-based methods (in particular the Kamila and LCM packages) and K-prototypes typically performed best in the setting of heterogeneous data.

scholarly journals A Multi-Branch U-Net for Steel Surface Defect Type and Severity Segmentation

Metals ◽  
2021 ◽  
Vol 11 (6) ◽  
pp. 870
Author(s):  
Robby Neven ◽  
Toon Goedemé
Keyword(s):  
Process Parameters ◽  
Visual Inspection ◽  
Steel Surface ◽  
Surface Defect ◽  
State Of The Art ◽  
Sheet Steel ◽  
Real Life ◽  
Single Task ◽  
Severity Estimation ◽  
Task Model

Automating sheet steel visual inspection can improve quality and reduce costs during its production. While many manufacturers still rely on manual or traditional inspection methods, deep learning-based approaches have proven their efficiency. In this paper, we go beyond the state-of-the-art in this domain by proposing a multi-task model that performs both pixel-based defect segmentation and severity estimation of the defects in one two-branch network. Additionally, we show how incorporation of the production process parameters improves the model’s performance. After manually constructing a real-life industrial dataset, we first implemented and trained two single-task models performing the defect segmentation and severity estimation tasks separately. Next, we compared this to a multi-task model that simultaneously performs the two tasks at hand. By combining the tasks into one model, both segmentation tasks improved by 2.5% and 3% mIoU, respectively. In the next step, we extended the multi-task model using sensor fusion with process parameters. We demonstrate that the incorporation of the process parameters resulted in a further mIoU increase of 6.8% and 2.9% for the defect segmentation and severity estimation tasks, respectively.

Sign in / Sign up

Export Citation Format

Share Document