A Novel End-to-End Model for Steering behavior Prediction of Autonomous Ego-Vehicles using Spatial and Temporal Attention Mechanism

2021 ◽  
Author(s):  
Lei Han ◽  
Lei Wu ◽  
Fujian Liang ◽  
Hao Cao ◽  
Dabing Luo ◽  
...  
Keyword(s):  
Attention Mechanism ◽  
Behavior Prediction ◽  
Temporal Attention ◽  
End To End

Multi-Head Spatio-Temporal Attention Mechanism for Urban Anomaly Event Prediction

2021 ◽  
Vol 5 (3) ◽  
pp. 1-21
Author(s):  
Huiqun Huang ◽  
Xi Yang ◽  
Suining He
Keyword(s):  
New York ◽  
Experimental Studies ◽  
Attention Mechanism ◽  
City Management ◽  
Temporal Attention ◽  
Spatial Correlations ◽  
Time Step ◽  
Event Prediction ◽  
Spatio Temporal ◽  
Prediction Approach

Timely forecasting the urban anomaly events in advance is of great importance to the city management and planning. However, anomaly event prediction is highly challenging due to the sparseness of data, geographic heterogeneity (e.g., complex spatial correlation, skewed spatial distribution of anomaly events and crowd flows), and the dynamic temporal dependencies. In this study, we propose M-STAP, a novel Multi-head Spatio-Temporal Attention Prediction approach to address the problem of multi-region urban anomaly event prediction. Specifically, M-STAP considers the problem from three main aspects: (1) extracting the spatial characteristics of the anomaly events in different regions, and the spatial correlations between anomaly events and crowd flows; (2) modeling the impacts of crowd flow dynamic of the most relevant regions in each time step on the anomaly events; and (3) employing attention mechanism to analyze the varying impacts of the historical anomaly events on the predicted data. We have conducted extensive experimental studies on the crowd flows and anomaly events data of New York City, Melbourne and Chicago. Our proposed model shows higher accuracy (41.91% improvement on average) in predicting multi-region anomaly events compared with the state-of-the-arts.

scholarly journals Research on Target Tracking Algorithm Based on Siamese Neural Network

2021 ◽  
Vol 2021 ◽  
pp. 1-11
Author(s):  
Haibo Pang ◽  
Qi Xuan ◽  
Meiqin Xie ◽  
Chengming Liu ◽  
Zhanbo Li
Keyword(s):  
Neural Network ◽  
Target Tracking ◽  
Evaluation Criteria ◽  
Attention Mechanism ◽  
Tracking Algorithm ◽  
Temporal Attention ◽  
Tracking Accuracy ◽  
Current Frame ◽  
Tracking Process ◽  
Recognition Ability

Target tracking is a significant topic in the field of computer vision. In this paper, the target tracking algorithm based on deep Siamese network is studied. Aiming at the situation that the tracking process is not robust, such as drift or miss the target, the tracking accuracy and robustness of the algorithm are improved by improving the feature extraction part and online update part. This paper adds SE-block and temporal attention mechanism (TAM) to the framework of Siamese neural network. SE-block can refine and extract features; different channels are given different weights according to their importance which can improve the discrimination of the network and the recognition ability of the tracker. Temporal attention mechanism can update the target state by adjusting the weights of samples at current frame and historical frame to solve the model drift caused by the existence of similar background. We use cross-entropy loss to distinguish the targets in different sequences so that their distance in the feature domains is longer and the features are easier to identify. We train and test the network on three benchmarks and compare with several state-of-the-art tracking methods. The experimental results demonstrate that the algorithm proposed is superior to other methods in tracking effect diagram and evaluation criteria. The proposed algorithm can solve the occlusion problem effectively while ensuring the real-time performance in the process of tracking.

scholarly journals Improving Hybrid CTC/Attention Architecture with Time-Restricted Self-Attention CTC for End-to-End Speech Recognition

2019 ◽  
Vol 9 (21) ◽  
pp. 4639 ◽  
Author(s):  
Long Wu ◽  
Ta Li ◽  
Li Wang ◽  
Yonghong Yan
Keyword(s):  
Speech Recognition ◽  
Proper Time ◽  
Window Size ◽  
Attention Mechanism ◽  
Wall Street ◽  
Word Error Rate ◽  
Perceptual Evaluation ◽  
Left And Right ◽  
End To End ◽  
Multiparty Interaction

As demonstrated in hybrid connectionist temporal classification (CTC)/Attention architecture, joint training with a CTC objective is very effective to solve the misalignment problem existing in the attention-based end-to-end automatic speech recognition (ASR) framework. However, the CTC output relies only on the current input, which leads to the hard alignment issue. To address this problem, this paper proposes the time-restricted attention CTC/Attention architecture, which integrates an attention mechanism with the CTC branch. “Time-restricted” means that the attention mechanism is conducted on a limited window of frames to the left and right. In this study, we first explore time-restricted location-aware attention CTC/Attention, establishing the proper time-restricted attention window size. Inspired by the success of self-attention in machine translation, we further introduce the time-restricted self-attention CTC/Attention that can better model the long-range dependencies among the frames. Experiments with wall street journal (WSJ), augmented multiparty interaction (AMI), and switchboard (SWBD) tasks demonstrate the effectiveness of the proposed time-restricted self-attention CTC/Attention. Finally, to explore the robustness of this method to noise and reverberation, we join a train neural beamformer frontend with the time-restricted attention CTC/Attention ASR backend in the CHIME-4 dataset. The reduction of word error rate (WER) and the increase of perceptual evaluation of speech quality (PESQ) approve the effectiveness of this framework.

STAT: Spatial-Temporal Attention Mechanism for Video Captioning

2020 ◽  
Vol 22 (1) ◽  
pp. 229-241 ◽  
Author(s):  
Chenggang Yan ◽  
Yunbin Tu ◽  
Xingzheng Wang ◽  
Yongbing Zhang ◽  
Xinhong Hao ◽  
...  
Keyword(s):  
Attention Mechanism ◽  
Temporal Attention ◽  
Video Captioning

scholarly journals A new joint CTC-attention-based speech recognition model with multi-level multi-head attention

2019 ◽  
Vol 2019 (1) ◽  
Author(s):  
Chu-Xiong Qin ◽  
Wen-Lin Zhang ◽  
Dan Qu
Keyword(s):  
Speech Recognition ◽  
Nonnegative Matrix Factorization ◽  
State Of The Art ◽  
Nonnegative Matrix ◽  
Attention Mechanism ◽  
Word Error Rate ◽  
Absolute Value ◽  
Multi Level ◽  
End To End ◽  
High Level

Abstract A method called joint connectionist temporal classification (CTC)-attention-based speech recognition has recently received increasing focus and has achieved impressive performance. A hybrid end-to-end architecture that adds an extra CTC loss to the attention-based model could force extra restrictions on alignments. To explore better the end-to-end models, we propose improvements to the feature extraction and attention mechanism. First, we introduce a joint model trained with nonnegative matrix factorization (NMF)-based high-level features. Then, we put forward a hybrid attention mechanism by incorporating multi-head attentions and calculating attention scores over multi-level outputs. Experiments on TIMIT indicate that the new method achieves state-of-the-art performance with our best model. Experiments on WSJ show that our method exhibits a word error rate (WER) that is only 0.2% worse in absolute value than the best referenced method, which is trained on a much larger dataset, and it beats all present end-to-end methods. Further experiments on LibriSpeech show that our method is also comparable to the state-of-the-art end-to-end system in WER.

Sign in / Sign up

Export Citation Format

Share Document