scholarly journals Key n -Gram Extractions and Analyses of Different Registers Based on Attention Network

2021 ◽  
Vol 2021 ◽  
pp. 1-16
Author(s):  
Haiyan Wu ◽  
Ying Liu ◽  
Shaoyun Shi ◽  
Qingfeng Wu ◽  
Yunlong Huang
Keyword(s):  
Feature Extraction ◽  
Attention Mechanism ◽  
Keyword Extraction ◽  
Model Design ◽  
Semantic Relationship ◽  
Baseline Model ◽  
Text Book ◽  
Advantages And Disadvantages ◽  
N Gram ◽  
Extraction Model

Key n -gram extraction can be seen as extracting n -grams which can distinguish different registers. Keyword (as n = 1 , 1-gram is the keyword) extraction models are generally carried out from two aspects, the feature extraction and the model design. By summarizing the advantages and disadvantages of existing models, we propose a novel key n -gram extraction model “attentive n -gram network” (ANN) based on the attention mechanism and multilayer perceptron, in which the attention mechanism scores each n -gram in a sentence by mining the internal semantic relationship between words, and their importance is given by the scores. Experimental results on the real corpus show that the key n -gram extracted from our model can distinguish a novel, news, and text book very well; the accuracy of our model is significantly higher than the baseline model. Also, we conduct experiments on key n -grams extracted from these registers, which turned out to be well clustered. Furthermore, we make some statistical analyses of the results of key n -gram extraction. We find that the key n -grams extracted by our model are very explanatory in linguistics.

scholarly journals Target Detection Network for SAR Images Based on Semi-Supervised Learning and Attention Mechanism

2021 ◽  
Vol 13 (14) ◽  
pp. 2686
Author(s):  
Di Wei ◽  
Yuang Du ◽  
Lan Du ◽  
Lu Li
Keyword(s):  
Feature Extraction ◽  
Supervised Learning ◽  
Target Detection ◽  
Scene Recognition ◽  
Attention Mechanism ◽  
Sar Image ◽  
Target Level ◽  
Sar Images ◽  
Training Samples ◽  
Pass Through

The existing Synthetic Aperture Radar (SAR) image target detection methods based on convolutional neural networks (CNNs) have achieved remarkable performance, but these methods require a large number of target-level labeled training samples to train the network. Moreover, some clutter is very similar to targets in SAR images with complex scenes, making the target detection task very difficult. Therefore, a SAR target detection network based on a semi-supervised learning and attention mechanism is proposed in this paper. Since the image-level label simply marks whether the image contains the target of interest or not, which is easier to be labeled than the target-level label, the proposed method uses a small number of target-level labeled training samples and a large number of image-level labeled training samples to train the network with a semi-supervised learning algorithm. The proposed network consists of a detection branch and a scene recognition branch with a feature extraction module and an attention module shared between these two branches. The feature extraction module can extract the deep features of the input SAR images, and the attention module can guide the network to focus on the target of interest while suppressing the clutter. During the semi-supervised learning process, the target-level labeled training samples will pass through the detection branch, while the image-level labeled training samples will pass through the scene recognition branch. During the test process, considering the help of global scene information in SAR images for detection, a novel coarse-to-fine detection procedure is proposed. After the coarse scene recognition determining whether the input SAR image contains the target of interest or not, the fine target detection is performed on the image that may contain the target. The experimental results based on the measured SAR dataset demonstrate that the proposed method can achieve better performance than the existing methods.

scholarly journals Offensive keyword extraction based on the attention mechanism of BERT and the eigenvector centrality using a graph representation

2021 ◽  
Author(s):  
Gretel Liz De la Peña Sarracén ◽  
Paolo Rosso
Keyword(s):  
Language Model ◽  
Hybrid Approach ◽  
Attention Mechanism ◽  
Graph Representation ◽  
Keyword Extraction ◽  
Eigenvector Centrality ◽  
Model Learning ◽  
Harmful Content ◽  
The Impact ◽  
Offensive Language

AbstractThe proliferation of harmful content on social media affects a large part of the user community. Therefore, several approaches have emerged to control this phenomenon automatically. However, this is still a quite challenging task. In this paper, we explore the offensive language as a particular case of harmful content and focus our study in the analysis of keywords in available datasets composed of offensive tweets. Thus, we aim to identify relevant words in those datasets and analyze how they can affect model learning. For keyword extraction, we propose an unsupervised hybrid approach which combines the multi-head self-attention of BERT and a reasoning on a word graph. The attention mechanism allows to capture relationships among words in a context, while a language model is learned. Then, the relationships are used to generate a graph from what we identify the most relevant words by using the eigenvector centrality. Experiments were performed by means of two mechanisms. On the one hand, we used an information retrieval system to evaluate the impact of the keywords in recovering offensive tweets from a dataset. On the other hand, we evaluated a keyword-based model for offensive language detection. Results highlight some points to consider when training models with available datasets.

scholarly journals A Multi-Branch Feature Fusion Strategy Based on an Attention Mechanism for Remote Sensing Image Scene Classification

2021 ◽  
Vol 13 (10) ◽  
pp. 1950
Author(s):  
Cuiping Shi ◽  
Xin Zhao ◽  
Liguo Wang
Keyword(s):  
Remote Sensing ◽  
Feature Extraction ◽  
Classification Accuracy ◽  
Feature Fusion ◽  
State Of The Art ◽  
Rapid Development ◽  
Remote Sensing Image ◽  
Classification Performance ◽  
Attention Mechanism ◽  
Scene Classification

In recent years, with the rapid development of computer vision, increasing attention has been paid to remote sensing image scene classification. To improve the classification performance, many studies have increased the depth of convolutional neural networks (CNNs) and expanded the width of the network to extract more deep features, thereby increasing the complexity of the model. To solve this problem, in this paper, we propose a lightweight convolutional neural network based on attention-oriented multi-branch feature fusion (AMB-CNN) for remote sensing image scene classification. Firstly, we propose two convolution combination modules for feature extraction, through which the deep features of images can be fully extracted with multi convolution cooperation. Then, the weights of the feature are calculated, and the extracted deep features are sent to the attention mechanism for further feature extraction. Next, all of the extracted features are fused by multiple branches. Finally, depth separable convolution and asymmetric convolution are implemented to greatly reduce the number of parameters. The experimental results show that, compared with some state-of-the-art methods, the proposed method still has a great advantage in classification accuracy with very few parameters.

scholarly journals On Orthogonal Feature Extraction model with applications in medical prognosis

2016 ◽  
Vol 40 (19-20) ◽  
pp. 8766-8776 ◽  
Author(s):  
Hao Jiang ◽  
Wai-Ki Ching ◽  
Wenpin Hou
Keyword(s):  
Feature Extraction ◽  
Extraction Model

scholarly journals Large-Scale Linear RankSVM

2014 ◽  
Vol 26 (4) ◽  
pp. 781-817 ◽  
Author(s):  
Ching-Pei Lee ◽  
Chih-Jen Lin
Keyword(s):  
Decision Trees ◽  
Computational Efficiency ◽  
Efficient Algorithm ◽  
Large Scale ◽  
Learning To Rank ◽  
Gradient Boosting ◽  
Baseline Model ◽  
Nonlinear Methods ◽  
Advantages And Disadvantages ◽  
Linear Ranksvm

Linear rankSVM is one of the widely used methods for learning to rank. Although its performance may be inferior to nonlinear methods such as kernel rankSVM and gradient boosting decision trees, linear rankSVM is useful to quickly produce a baseline model. Furthermore, following its recent development for classification, linear rankSVM may give competitive performance for large and sparse data. A great deal of works have studied linear rankSVM. The focus is on the computational efficiency when the number of preference pairs is large. In this letter, we systematically study existing works, discuss their advantages and disadvantages, and propose an efficient algorithm. We discuss different implementation issues and extensions with detailed experiments. Finally, we develop a robust linear rankSVM tool for public use.

scholarly journals Unsupervised Text Feature Extraction for Academic Chatbot using Constrained FP-Growth

2021 ◽  
Vol 14 ◽  
pp. 1-11
Author(s):  
Suraya Alias
Keyword(s):  
Feature Extraction ◽  
Customer Service ◽  
User Satisfaction ◽  
Feature Vector ◽  
Size Reduction ◽  
Growth Technique ◽  
Feature Extraction Method ◽  
Text Feature ◽  
Text Features ◽  
N Gram

In the edge where conversation merely involves online chatting and texting one another, an automated conversational agent is needed to support certain repetitive tasks such as providing FAQs, customer service and product recommendations. One of the key challenges is to identify and discover user’s intention in a social conversation where the focus of our work in the academic domain. Our unsupervised text feature extraction method for Intent Pattern Discovery is developed by applying text features constraints to the FP-Growth technique. The academic corpus was developed using a chat messages dataset where the conversation between students and academicians regarding undergraduate and postgraduate queries were extracted as text features for our model. We experimented with our new Constrained Frequent Intent Pattern (cFIP) model in contrast with the N-gram model in terms of feature-vector size reduction, descriptive intent discovery, and analysis of cFIP Rules. Our findings show significant and descriptive intent patterns was discovered with confidence rules value of 0.9 for cFIP of 3-sequence. We report an average feature-vector size reduction of 76% compared to the Bigram model using both undergraduate and postgraduate conversation datasets. The usability testing results depicted overall user satisfaction average mean score is 4.30 out of 5 in using the Academic chatbot which supported our intent discovery cFIP approach.

Sign in / Sign up

Export Citation Format

Share Document