scholarly journals Knowledge dialogue generation with multi-head attention mechanism

2021 ◽  
Vol 2078 (1) ◽  
pp. 012013
Author(s):  
Fengrui Yang ◽  
Xiaosi Wu ◽  
Luwei Zhao
Keyword(s):  
Attention Mechanism ◽  
Baseline Model ◽  
User Input ◽  
Knowledge Diversity ◽  
Current Input ◽  
Increasing Knowledge ◽  
Knowledge Content

Abstract An increasing number of research efforts are focusing on knowledge dialogue generation. Less attention is focused on increasing knowledge diversity in generated responses. A model of knowledge selection guided by a multi-head attention mechanism is proposed. First, the current input discourse and knowledge content are input into the Bi-GRU module to obtain the coding vector, and then obtain multiple aspects of semantics from the user input discourse coding vector based on the multi-head attention mechanism, so as to select different knowledge. A punishment item method is proposed to force different attention to focus on different aspects, and finally, use the user input and selected knowledge for the decoding stage. Experiments with manual and automated evaluations have proven that the model is superior to the baseline model compared to previous work.

scholarly journals Key n -Gram Extractions and Analyses of Different Registers Based on Attention Network

2021 ◽  
Vol 2021 ◽  
pp. 1-16
Author(s):  
Haiyan Wu ◽  
Ying Liu ◽  
Shaoyun Shi ◽  
Qingfeng Wu ◽  
Yunlong Huang
Keyword(s):  
Feature Extraction ◽  
Attention Mechanism ◽  
Keyword Extraction ◽  
Model Design ◽  
Semantic Relationship ◽  
Baseline Model ◽  
Text Book ◽  
Advantages And Disadvantages ◽  
N Gram ◽  
Extraction Model

Key n -gram extraction can be seen as extracting n -grams which can distinguish different registers. Keyword (as n = 1 , 1-gram is the keyword) extraction models are generally carried out from two aspects, the feature extraction and the model design. By summarizing the advantages and disadvantages of existing models, we propose a novel key n -gram extraction model “attentive n -gram network” (ANN) based on the attention mechanism and multilayer perceptron, in which the attention mechanism scores each n -gram in a sentence by mining the internal semantic relationship between words, and their importance is given by the scores. Experimental results on the real corpus show that the key n -gram extracted from our model can distinguish a novel, news, and text book very well; the accuracy of our model is significantly higher than the baseline model. Also, we conduct experiments on key n -grams extracted from these registers, which turned out to be well clustered. Furthermore, we make some statistical analyses of the results of key n -gram extraction. We find that the key n -grams extracted by our model are very explanatory in linguistics.

A Model for Structured Data Entry Based on Explicit Descriptional Knowledge

1994 ◽  
Vol 33 (05) ◽  
pp. 454-463 ◽  
Author(s):  
A. M. van Ginneken ◽  
J. van der Lei ◽  
J. H. van Bemmel ◽  
P. W. Moorman
Keyword(s):  
Domain Knowledge ◽  
Data Entry ◽  
Knowledge Bases ◽  
Structured Data ◽  
Research Quality ◽  
Free Text ◽  
Specific Knowledge ◽  
User Input ◽  
The One ◽  
Research Quality Assessment

Abstract:Clinical narratives in patient records are usually recorded in free text, limiting the use of this information for research, quality assessment, and decision support. This study focuses on the capture of clinical narratives in a structured format by supporting physicians with structured data entry (SDE). We analyzed and made explicit which requirements SDE should meet to be acceptable for the physician on the one hand, and generate unambiguous patient data on the other. Starting from these requirements, we found that in order to support SDE, the knowledge on which it is based needs to be made explicit: we refer to this knowledge as descriptional knowledge. We articulate the nature of this knowledge, and propose a model in which it can be formally represented. The model allows the construction of specific knowledge bases, each representing the knowledge needed to support SDE within a circumscribed domain. Data entry is made possible through a general entry program, of which the behavior is determined by a combination of user input and the content of the applicable domain knowledge base. We clarify how descriptional knowledge is represented, modeled, and used for data entry to achieve SDE, which meets the proposed requirements.

An Attention Mechanism Extension of Automatic HTML Generation from Web Page Design Images

2020 ◽  
Vol 140 (12) ◽  
pp. 1393-1401
Author(s):  
Hiroki Chinen ◽  
Hidehiro Ohki ◽  
Keiji Gyohten ◽  
Toshiya Takami
Keyword(s):  
Attention Mechanism ◽  
Web Page ◽  
Web Page Design ◽  
Page Design

The Effect of Using Ramps with Trench Cylindrical Holes on Film Cooling Effectiveness

2015 ◽  
Vol 3 (2) ◽  
pp. 15-27
Author(s):  
Ahmed A. Imram ◽  
Humam K. Jalghef ◽  
Falah F. Hatem
Keyword(s):  
Numerical Simulation ◽  
Film Cooling ◽  
Air Injection ◽  
Baseline Model ◽  
Cooling Effectiveness ◽  
Film Cooling Effectiveness ◽  
Blowing Ratio ◽  
Hot Air ◽  
Test Study ◽  
Cylindrical Holes

     The effect of introducing ramp with a cylindrical slot hole on the film cooling effectiveness has been investigated experimentally and numerically. The film cooling effectiveness measurements are obtained experimentally. A test study was performed at a single mainstream with Reynolds number 76600 at three different coolant to mainstream blowing ratios 1.5, 2, and 3. Numerical simulation is introduced to primarily estimate the best ramp configurations and to predict the behavior of the transport phenomena in the region linked closely to the interaction between the coolant air injection and the hot air mainstram flow. The results showed that using ramps with trench cylindrical holes would enhanced the overall film cooling effectiveness by 83.33% compared with baseline model at blowing ratio of 1.5, also  the best overall flim cooling effectevness was obtained at blowing ratio of 2 while it is reduced at blowing ratio of 3.

scholarly journals Dam Deformation Interpretation and Prediction Based on a Long Short-Term Memory Model Coupled with an Attention Mechanism

2021 ◽  
Vol 11 (14) ◽  
pp. 6625
Author(s):  
Yan Su ◽  
Kailiang Weng ◽  
Chuan Lin ◽  
Zeqin Chen
Keyword(s):  
Short Term Memory ◽  
Attention Mechanism ◽  
Impact Factors ◽  
Nonlinear Prediction ◽  
Time Dimension ◽  
Short Term ◽  
Deformation Prediction ◽  
Term Memory ◽  
Long Short Term Memory ◽  
Dam Deformation

An accurate dam deformation prediction model is vital to a dam safety monitoring system, as it helps assess and manage dam risks. Most traditional dam deformation prediction algorithms ignore the interpretation and evaluation of variables and lack qualitative measures. This paper proposes a data processing framework that uses a long short-term memory (LSTM) model coupled with an attention mechanism to predict the deformation response of a dam structure. First, the random forest (RF) model is introduced to assess the relative importance of impact factors and screen input variables. Secondly, the density-based spatial clustering of applications with noise (DBSCAN) method is used to identify and filter the equipment based abnormal values to reduce the random error in the measurements. Finally, the coupled model is used to focus on important factors in the time dimension in order to obtain more accurate nonlinear prediction results. The results of the case study show that, of all tested methods, the proposed coupled method performed best. In addition, it was found that temperature and water level both have significant impacts on dam deformation and can serve as reliable metrics for dam management.

Sign in / Sign up

Export Citation Format

Share Document