scholarly journals Learning Sentence Representation with Guidance of Human Attention

2017 ◽  
Author(s):  
Shaonan Wang ◽  
Jiajun Zhang ◽  
Chengqing Zong
Keyword(s):  
Reading Time ◽  
State Of The Art ◽  
General Purpose ◽  
The State ◽  
Attention Mechanism ◽  
Human Attention ◽  
Made In

Recently, much progress has been made in learning general-purpose sentence representations that can be used across domains. However, most of the existing models typically treat each word in a sentence equally. In contrast, extensive studies have proven that human read sentences efficiently by making a sequence of fixation and saccades. This motivates us to improve sentence representations by assigning different weights to the vectors of the component words, which can be treated as an attention mechanism on single sentences. To that end, we propose two novel attention models, in which the attention weights are derived using significant predictors of human reading time, i.e., Surprisal, POS tags and CCG supertags. The extensive experiments demonstrate that the proposed methods significantly improve upon the state-of-the-art sentence representation models.

A Comprehensive Taxonomy of Dynamic Texture Representation

2023 ◽  
Vol 55 (1) ◽  
pp. 1-39
Author(s):  
Thanh Tuan Nguyen ◽  
Thanh Phuong Nguyen
Keyword(s):  
Large Scale ◽  
Environmental Changes ◽  
State Of The Art ◽  
The State ◽  
Future Research ◽  
Research Activities ◽  
Potential Applications ◽  
Benchmark Datasets ◽  
Negative Impacts ◽  
Made In

Representing dynamic textures (DTs) plays an important role in many real implementations in the computer vision community. Due to the turbulent and non-directional motions of DTs along with the negative impacts of different factors (e.g., environmental changes, noise, illumination, etc.), efficiently analyzing DTs has raised considerable challenges for the state-of-the-art approaches. For 20 years, many different techniques have been introduced to handle the above well-known issues for enhancing the performance. Those methods have shown valuable contributions, but the problems have been incompletely dealt with, particularly recognizing DTs on large-scale datasets. In this article, we present a comprehensive taxonomy of DT representation in order to purposefully give a thorough overview of the existing methods along with overall evaluations of their obtained performances. Accordingly, we arrange the methods into six canonical categories. Each of them is then taken in a brief presentation of its principal methodology stream and various related variants. The effectiveness levels of the state-of-the-art methods are then investigated and thoroughly discussed with respect to quantitative and qualitative evaluations in classifying DTs on benchmark datasets. Finally, we point out several potential applications and the remaining challenges that should be addressed in further directions. In comparison with two existing shallow DT surveys (i.e., the first one is out of date as it was made in 2005, while the newer one (published in 2016) is an inadequate overview), we believe that our proposed comprehensive taxonomy not only provides a better view of DT representation for the target readers but also stimulates future research activities.

SANTM: Efficient Self-attention-driven Network for Text Matching

2022 ◽  
Vol 22 (3) ◽  
pp. 1-21
Author(s):  
Prayag Tiwari ◽  
Amit Kumar Jaiswal ◽  
Sahil Garg ◽  
Ilsun You
Keyword(s):  
Natural Language ◽  
State Of The Art ◽  
The State ◽  
Attention Mechanism ◽  
Matching Problems ◽  
Attention Model ◽  
Extra Information ◽  
Textual Entailment ◽  
Benchmark Datasets ◽  
Text Matching

Self-attention mechanisms have recently been embraced for a broad range of text-matching applications. Self-attention model takes only one sentence as an input with no extra information, i.e., one can utilize the final hidden state or pooling. However, text-matching problems can be interpreted either in symmetrical or asymmetrical scopes. For instance, paraphrase detection is an asymmetrical task, while textual entailment classification and question-answer matching are considered asymmetrical tasks. In this article, we leverage attractive properties of self-attention mechanism and proposes an attention-based network that incorporates three key components for inter-sequence attention: global pointwise features, preceding attentive features, and contextual features while updating the rest of the components. Our model follows evaluation on two benchmark datasets cover tasks of textual entailment and question-answer matching. The proposed efficient Self-attention-driven Network for Text Matching outperforms the state of the art on the Stanford Natural Language Inference and WikiQA datasets with much fewer parameters.

The Technique of Raman Spectroscopy: A State-of-the-Art Comparison to Infrared

1971 ◽  
Vol 25 (4) ◽  
pp. 430-439 ◽  
Author(s):  
Howard J. Sloane
Keyword(s):  
Raman Spectroscopy ◽  
State Of The Art ◽  
The State ◽  
Point Of View ◽  
Complementary Information ◽  
Future Outlook ◽  
Sample Handling ◽  
Advantages And Disadvantages ◽  
Sampling Procedures ◽  
Made In

This paper in a tabulated summary format discusses the state-of-the-art of Raman spectroscopy for commercially available instrumentation. A comparison to infrared is made in terms of (I) instrumentation, (II) sample handling, and (III) applications. Although the two techniques yield similar and often complementary information, they are quite different from the point of view of instrumentation and sampling procedures. This leads to various advantages and disadvantages or limitations for each. These are discussed as well as the future outlook.

scholarly journals A Span-based Joint Model for Opinion Target Extraction and Target Sentiment Classification

2019 ◽  
Author(s):  
Yan Zhou ◽  
Longtao Huang ◽  
Tao Guo ◽  
Jizhong Han ◽  
Songlin Hu
Keyword(s):  
Sentiment Analysis ◽  
State Of The Art ◽  
Joint Model ◽  
The State ◽  
Attention Mechanism ◽  
Sentiment Classification ◽  
Global Information ◽  
Target Extraction ◽  
Benchmark Datasets ◽  
Tagging Methods

Target-Based Sentiment Analysis aims at extracting opinion targets and classifying the sentiment polarities expressed on each target. Recently, token based sequence tagging methods have been successfully applied to jointly solve the two tasks, which aims to predict a tag for each token. Since they do not treat a target containing several words as a whole, it might be difficult to make use of the global information to identify that opinion target, leading to incorrect extraction. Independently predicting the sentiment for each token may also lead to sentiment inconsistency for different words in an opinion target. In this paper, inspired by span-based methods in NLP, we propose a simple and effective joint model to conduct extraction and classification at span level rather than token level. Our model first emulates spans with one or more tokens and learns their representation based on the tokens inside. And then, a span-aware attention mechanism is designed to compute the sentiment information towards each span. Extensive experiments on three benchmark datasets show that our model consistently outperforms the state-of-the-art methods.

scholarly journals An Extended Commentary on Post-Publication Peer Review In Organizational Neuroscience

2018 ◽  
Author(s):  
Guy A Prochilo ◽  
Winnifred R Louis ◽  
Stefan Bode ◽  
Hannes Zacher ◽  
Pascal Molenberghs
Keyword(s):  
Peer Review ◽  
State Of The Art ◽  
Empirical Work ◽  
The State ◽  
Statistical Testing ◽  
The Past ◽  
Explicit Reference ◽  
Major Review ◽  
Specific Concern ◽  
Made In

Note: this manuscript has been peer reviewed and is published in Meta-Psychology. Please cite as: Prochilo, G. A., Louis, W. R., Bode, S., Zacher, H., & Molenberghs, P. (2019). An Extended Commentary on Post-publication Peer Review in Organizational Neuroscience. Meta-Psychology, 3. https://doi.org/10.15626/MP.2018.935 | While considerable progress has been made in organizational neuroscience over the past decade, we argue that critical evaluations of published empirical works are not being conducted carefully and consistently. In this extended commentary we take as an example Waldman and colleagues (2017): a major review work that evaluates the state-of-the-art of organizational neuroscience. In what should be an evaluation of the field’s empirical work, the authors uncritically summarize a series of studies that: (1) provide insufficient transparency to be clearly understood, evaluated, or replicated, and/or (2) which misuse inferential tests that lead to misleading conclusions, among other concerns. These concerns have been ignored across multiple major reviews and citing articles. We therefore provide a post-publication review (in two parts) of one-third of all studies evaluated in Waldman and colleague’s major review work. In Part I, we systematically evaluate the field’s two seminal works with respect to their methods, analytic strategy, results, and interpretation of findings. And in Part II, we provide focused reviews of secondary works that each center on a specific concern we suggest should be a point of discussion as the field moves forward. In doing so, we identify a series of practices we recommend will improve the state of the literature. This includes: (1) evaluating the transparency and completeness of an empirical article before accepting its claims, (2) becoming familiar with common misuses or misconceptions of statistical testing, and (3) interpreting results with an explicit reference to effect size magnitude, precision, and accuracy, among other recommendations. We suggest that adopting these practices will motivate the development of a more replicable, reliable, and trustworthy field of organizational neuroscience moving forward.

scholarly journals Highway Travel Time Prediction of Segments Based on ANPR Data considering Traffic Diversion

2021 ◽  
Vol 2021 ◽  
pp. 1-16
Author(s):  
Wenjun Du ◽  
Bo Sun ◽  
Jiating Kuai ◽  
Jiemin Xie ◽  
Jie Yu ◽  
...  
Keyword(s):  
Travel Time ◽  
Traffic Management ◽  
State Of The Art ◽  
The State ◽  
Attention Mechanism ◽  
Critical Parameters ◽  
Depth Information ◽  
Highway Traffic ◽  
Residual Network ◽  
Traffic Diversion

Travel time is one of the most critical parameters in proactive traffic management and the deployment of advanced traveler information systems. This paper proposes a hybrid model named LSTM-CNN for predicting the travel time of highways by integrating the long short-term memory (LSTM) and the convolutional neural networks (CNNs) with the attention mechanism and the residual network. The highway is divided into multiple segments by considering the traffic diversion and the relative location of automatic number plate recognition (ANPR). There are four steps in this hybrid approach. First, the average travel time of each segment in each interval is calculated from ANPR and fed into LSTM in the form of a multidimensional array. Second, the attention mechanism is adopted to combine the hidden layer of LSTM with dynamic temporal weights. Third, the residual network is introduced to increase the network depth and overcome the vanishing gradient problem, which consists of three pairs of one-dimensional convolutional layers (Conv1D) and batch normalization (BatchNorm) with the rectified linear unit (ReLU) as the activation function. Finally, a series of Conv1D layers is connected to extract features further and reduce dimensionality. The proposed LSTM-CNN approach is tested on the three-month ANPR data of a real-world 39.25 km highway with four pairs of ANPR detectors of the uplink and downlink, Zhejiang, China. The experimental results indicate that LSTM-CNN learns spatial, temporal, and depth information better than the state-of-the-art traffic forecasting models, so LSTM-CNN can predict more accurate travel time. Moreover, LSTM-CNN outperforms the state-of-the-art methods in nonrecurrent prediction, multistep-ahead prediction, and long-term prediction. LSTM-CNN is a promising model with scalability and portability for highway traffic prediction and can be further extended to improve the performance of the advanced traffic management system (ATMS) and advanced traffic information system (ATIS).

scholarly journals Guiding Attention in Sequence-to-Sequence Models for Dialogue Act Prediction

2020 ◽  
Vol 34 (05) ◽  
pp. 7594-7601
Author(s):  
Pierre Colombo ◽  
Emile Chapuis ◽  
Matteo Manica ◽  
Emmanuel Vignon ◽  
Giovanna Varni ◽  
...  
Keyword(s):  
Machine Translation ◽  
Random Fields ◽  
Conditional Random Fields ◽  
State Of The Art ◽  
The State ◽  
Attention Mechanism ◽  
Accuracy Score ◽  
Beam Search ◽  
Conversational Agents ◽  
Neural Machine Translation

The task of predicting dialog acts (DA) based on conversational dialog is a key component in the development of conversational agents. Accurately predicting DAs requires a precise modeling of both the conversation and the global tag dependencies. We leverage seq2seq approaches widely adopted in Neural Machine Translation (NMT) to improve the modelling of tag sequentiality. Seq2seq models are known to learn complex global dependencies while currently proposed approaches using linear conditional random fields (CRF) only model local tag dependencies. In this work, we introduce a seq2seq model tailored for DA classification using: a hierarchical encoder, a novel guided attention mechanism and beam search applied to both training and inference. Compared to the state of the art our model does not require handcrafted features and is trained end-to-end. Furthermore, the proposed approach achieves an unmatched accuracy score of 85% on SwDA, and state-of-the-art accuracy score of 91.6% on MRDA.

ADVANCE PLANNING FOR DISPERSANT USE/NON-USE

1985 ◽  
Vol 1985 (1) ◽  
pp. 429-432 ◽  
Author(s):  
J. P. Fraser
Keyword(s):  
Oil Spill ◽  
Oil Spills ◽  
State Of The Art ◽  
Geographical Area ◽  
The State ◽  
Environmental Damage ◽  
Advance Planning ◽  
Made In

ABSTRACT Guidelines are suggested for advance planning for the use or non-use of dispersants to combat oil spills. These guidelines are intended to expedite the decision to use dispersants in the event of an oil spill, where that will minimize environmental damage. These guidelines can be applied readily to any geographical area to answer the following questions: (1) Are there locations where dispersant application should normally be allowed? (2) In these locations, what rate of dispersant application should be allowed? (3) Are there locations where dispersant application should normally be avoided? The logic behind these guidelines is explained so that exceptions can be identified and so that changes in the guidelines can be made as advances are made in the state of the art. These guidelines provide for control over dispersant usage while allowing application (in most instances) at rates which can disperse floating oil effectively.

Special Article: Death Anxiety Research: The State of the Art

1997 ◽  
Vol 36 (2) ◽  
pp. 97-120 ◽  
Author(s):  
Robert A. Neimeyer
Keyword(s):  
Death Anxiety ◽  
State Of The Art ◽  
The State ◽  
Future Research ◽  
Death Attitudes ◽  
Development And Validation ◽  
Made In

Despite forty years of research in death attitudes, our understanding of the causes, correlates, and consequences of death related anxieties and fears remains less than comprehensive. However, clear gains have been made in the measurement of death concerns and competencies, leading to the development and validation of a handful of scales whose more extensive use could improve the conceptual yield of research in this area. In this article, I review these promising approaches to the assessment of death attitudes, as well as a number of theoretical, methodological, and practical issues surrounding their use. If investigators devote equal attention to the quality and quantity of future research, there is reason to hope that psychology could make a more profound and systematic contribution to our understanding of the human encounter with death.

System reliability modelling: The current capability and potential future developments

2009 ◽  
Vol 223 (12) ◽  
pp. 2881-2897 ◽  
Author(s):  
J Andrews
Keyword(s):  
System Reliability ◽  
State Of The Art ◽  
Reliability Assessment ◽  
The State ◽  
Engineering Systems ◽  
Industrial Systems ◽  
Future Developments ◽  
Reliability Modelling ◽  
Assessment Techniques ◽  
Made In

This article describes the state-of-the-art methods available for systems reliability assessment. The significant contributions made to those methods in common use for the analysis of industrial systems are identified. The article reviews the developments in engineering systems that are likely to occur over the next decade that will challenge the current capability in this field and the potential advances that may result. A discussion is also provided of novel uses that will become possible due to the advances made in the assessment techniques over this period.

Sign in / Sign up

Export Citation Format

Share Document