BERT-BU12 Hate Speech Detection using Bidirectional Encoder-Decoder

2022 ◽  
Vol 11 (2) ◽  
pp. 0-0
Keyword(s):  
Text Classification ◽  
Question Answering ◽  
Hate Speech ◽  
State Of The Art ◽  
Learning Models ◽  
Speech Detection ◽  
Attention Networks ◽  
Attention Model ◽  
Proposed Model ◽  
Novel Method

In the recent times transfer learning models have known to exhibited good results in the area of text classification for question-answering, summarization, next word prediction but these learning models have not been extensively used for the problem of hate speech detection yet. We anticipate that these networks may give better results in another task of text classification i.e. hate speech detection. This paper introduces a novel method of hate speech detection based on the concept of attention networks using the BERT attention model. We have conducted exhaustive experiments and evaluation over publicly available datasets using various evaluation metrics (precision, recall and F1 score). We show that our model outperforms all the state-of-the-art methods by almost 4%. We have also discussed in detail the technical challenges faced during the implementation of the proposed model.

scholarly journals A Web Interface for Analyzing Hate Speech

2021 ◽  
Vol 13 (3) ◽  
pp. 80
Author(s):  
Lazaros Vrysis ◽  
Nikolaos Vryzas ◽  
Rigas Kotsakis ◽  
Theodora Saridou ◽  
Maria Matsiola ◽  
...  
Keyword(s):  
Machine Learning ◽  
Social Media ◽  
Graphical User Interface ◽  
Hate Speech ◽  
Web Interface ◽  
Learning Models ◽  
Speech Detection ◽  
Media Services ◽  
The Web ◽  
Machine Learning Models

Social media services make it possible for an increasing number of people to express their opinion publicly. In this context, large amounts of hateful comments are published daily. The PHARM project aims at monitoring and modeling hate speech against refugees and migrants in Greece, Italy, and Spain. In this direction, a web interface for the creation and the query of a multi-source database containing hate speech-related content is implemented and evaluated. The selected sources include Twitter, YouTube, and Facebook comments and posts, as well as comments and articles from a selected list of websites. The interface allows users to search in the existing database, scrape social media using keywords, annotate records through a dedicated platform and contribute new content to the database. Furthermore, the functionality for hate speech detection and sentiment analysis of texts is provided, making use of novel methods and machine learning models. The interface can be accessed online with a graphical user interface compatible with modern internet browsers. For the evaluation of the interface, a multifactor questionnaire was formulated, targeting to record the users’ opinions about the web interface and the corresponding functionality.

scholarly journals To BAN or Not to BAN: Bayesian Attention Networks for Reliable Hate Speech Detection

2021 ◽  
Author(s):  
Kristian Miok ◽  
Blaž Škrlj ◽  
Daniela Zaharie ◽  
Marko Robnik-Šikonja
Keyword(s):  
Monte Carlo ◽  
Hate Speech ◽  
Classification Performance ◽  
Reliability Estimation ◽  
Superior Performance ◽  
Speech Detection ◽  
Attention Networks ◽  
Reliability Estimates ◽  
Viable Mechanism ◽  
Affective Dimensions

AbstractHate speech is an important problem in the management of user-generated content. To remove offensive content or ban misbehaving users, content moderators need reliable hate speech detectors. Recently, deep neural networks based on the transformer architecture, such as the (multilingual) BERT model, have achieved superior performance in many natural language classification tasks, including hate speech detection. So far, these methods have not been able to quantify their output in terms of reliability. We propose a Bayesian method using Monte Carlo dropout within the attention layers of the transformer models to provide well-calibrated reliability estimates. We evaluate and visualize the results of the proposed approach on hate speech detection problems in several languages. Additionally, we test whether affective dimensions can enhance the information extracted by the BERT model in hate speech classification. Our experiments show that Monte Carlo dropout provides a viable mechanism for reliability estimation in transformer networks. Used within the BERT model, it offers state-of-the-art classification performance and can detect less trusted predictions.

scholarly journals Multi-agent Attentional Activity Recognition

2019 ◽  
Author(s):  
Kaixuan Chen ◽  
Lina Yao ◽  
Dalin Zhang ◽  
Bin Guo ◽  
Zhiwen Yu
Keyword(s):  
Activity Recognition ◽  
State Of The Art ◽  
Body Part ◽  
Body Parts ◽  
Temporal Attention ◽  
Attention Model ◽  
Proposed Model ◽  
Collective Motions ◽  
Multi Agent ◽  
Real World Datasets

Multi-modality is an important feature of sensor based activity recognition. In this work, we consider two inherent characteristics of human activities, the spatially-temporally varying salience of features and the relations between activities and corresponding body part motions. Based on these, we propose a multi-agent spatial-temporal attention model. The spatial-temporal attention mechanism helps intelligently select informative modalities and their active periods. And the multiple agents in the proposed model represent activities with collective motions across body parts by independently selecting modalities associated with single motions. With a joint recognition goal, the agents share gained information and coordinate their selection policies to learn the optimal recognition model. The experimental results on four real-world datasets demonstrate that the proposed model outperforms the state-of-the-art methods.

scholarly journals A recommendations model with multiaspect awareness and hierarchical user-product attention mechanisms

2020 ◽  
Vol 17 (3) ◽  
pp. 849-865
Author(s):  
Zhongqin Bi ◽  
Shuming Dou ◽  
Zhe Liu ◽  
Yongbin Li
Keyword(s):  
State Of The Art ◽  
Weight Vector ◽  
User Preferences ◽  
The Other ◽  
Attention Networks ◽  
Proposed Model ◽  
Network Methods ◽  
Public Datasets ◽  
Novel Model ◽  
Attention Weight

Neural network methods have been trained to satisfactorily learn user/product representations from textual reviews. A representation can be considered as a multiaspect attention weight vector. However, in several existing methods, it is assumed that the user representation remains unchanged even when the user interacts with products having diverse characteristics, which leads to inaccurate recommendations. To overcome this limitation, this paper proposes a novel model to capture the varying attention of a user for different products by using a multilayer attention framework. First, two individual hierarchical attention networks are used to encode the users and products to learn the user preferences and product characteristics from review texts. Then, we design an attention network to reflect the adaptive change in the user preferences for each aspect of the targeted product in terms of the rating and review. The results of experiments performed on three public datasets demonstrate that the proposed model notably outperforms the other state-of-the-art baselines, thereby validating the effectiveness of the proposed approach.

scholarly journals Enhancing the performance of cancer text classification model based on cancer hallmarks

2021 ◽  
Vol 10 (2) ◽  
pp. 316
Author(s):  
Noha Ali ◽  
Ahmed H. AbuEl-Atta ◽  
Hala H. Zayed
Keyword(s):  
Neural Network ◽  
Language Processing ◽  
Text Classification ◽  
State Of The Art ◽  
Classification Model ◽  
Biomedical Text ◽  
Cancer Hallmarks ◽  
Embedding Technique ◽  
Proposed Model ◽  
Biomedical Text Classification

<span id="docs-internal-guid-cb130a3a-7fff-3e11-ae3d-ad2310e265f8"><span>Deep learning (DL) algorithms achieved state-of-the-art performance in computer vision, speech recognition, and natural language processing (NLP). In this paper, we enhance the convolutional neural network (CNN) algorithm to classify cancer articles according to cancer hallmarks. The model implements a recent word embedding technique in the embedding layer. This technique uses the concept of distributed phrase representation and multi-word phrases embedding. The proposed model enhances the performance of the existing model used for biomedical text classification. The result of the proposed model overcomes the previous model by achieving an F-score equal to 83.87% using an unsupervised technique that trained on PubMed abstracts called PMC vectors (PMCVec) embedding. Also, we made another experiment on the same dataset using the recurrent neural network (RNN) algorithm with two different word embeddings Google news and PMCVec which achieving F-score equal to 74.9% and 76.26%, respectively.</span></span>

scholarly journals Sentiment Classification Using Convolutional Neural Networks

2019 ◽  
Vol 9 (11) ◽  
pp. 2347 ◽  
Author(s):  
Hannah Kim ◽  
Young-Seob Jeong
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Text Classification ◽  
State Of The Art ◽  
Sentiment Classification ◽  
Learning Models ◽  
Text Data ◽  
Textual Data ◽  
Better Than

As the number of textual data is exponentially increasing, it becomes more important to develop models to analyze the text data automatically. The texts may contain various labels such as gender, age, country, sentiment, and so forth. Using such labels may bring benefits to some industrial fields, so many studies of text classification have appeared. Recently, the Convolutional Neural Network (CNN) has been adopted for the task of text classification and has shown quite successful results. In this paper, we propose convolutional neural networks for the task of sentiment classification. Through experiments with three well-known datasets, we show that employing consecutive convolutional layers is effective for relatively longer texts, and our networks are better than other state-of-the-art deep learning models.

scholarly journals Convolutional Spatial Attention Model for Reading Comprehension with Multiple-Choice Questions

2019 ◽  
Vol 33 ◽  
pp. 6276-6283 ◽  
Author(s):  
Zhipeng Chen ◽  
Yiming Cui ◽  
Wentao Ma ◽  
Shijin Wang ◽  
Guoping Hu
Keyword(s):  
Reading Comprehension ◽  
Mutual Information ◽  
Spatial Attention ◽  
State Of The Art ◽  
Multiple Choice ◽  
Multiple Choice Questions ◽  
Attention Model ◽  
Novel Approach ◽  
Proposed Model ◽  
Machine Reading

Machine Reading Comprehension (MRC) with multiplechoice questions requires the machine to read given passage and select the correct answer among several candidates. In this paper, we propose a novel approach called Convolutional Spatial Attention (CSA) model which can better handle the MRC with multiple-choice questions. The proposed model could fully extract the mutual information among the passage, question, and the candidates, to form the enriched representations. Furthermore, to merge various attention results, we propose to use convolutional operation to dynamically summarize the attention values within the different size of regions. Experimental results show that the proposed model could give substantial improvements over various state-of- the-art systems on both RACE and SemEval-2018 Task11 datasets.

scholarly journals Beyond RNNs: Positional Self-Attention with Co-Attention for Video Question Answering

2019 ◽  
Vol 33 ◽  
pp. 8658-8665 ◽  
Author(s):  
Xiangpeng Li ◽  
Jingkuan Song ◽  
Lianli Gao ◽  
Xianglong Liu ◽  
Wenbing Huang ◽  
...  
Keyword(s):  
Question Answering ◽  
State Of The Art ◽  
Computation Time ◽  
Comparable Result ◽  
Video Encoding ◽  
Visual Question Answering ◽  
Proposed Model ◽  
Ablation Study ◽  
The Given ◽  
Video Question Answering

Most of the recent progresses on visual question answering are based on recurrent neural networks (RNNs) with attention. Despite the success, these models are often timeconsuming and having difficulties in modeling long range dependencies due to the sequential nature of RNNs. We propose a new architecture, Positional Self-Attention with Coattention (PSAC), which does not require RNNs for video question answering. Specifically, inspired by the success of self-attention in machine translation task, we propose a Positional Self-Attention to calculate the response at each position by attending to all positions within the same sequence, and then add representations of absolute positions. Therefore, PSAC can exploit the global dependencies of question and temporal information in the video, and make the process of question and video encoding executed in parallel. Furthermore, in addition to attending to the video features relevant to the given questions (i.e., video attention), we utilize the co-attention mechanism by simultaneously modeling “what words to listen to” (question attention). To the best of our knowledge, this is the first work of replacing RNNs with selfattention for the task of visual question answering. Experimental results of four tasks on the benchmark dataset show that our model significantly outperforms the state-of-the-art on three tasks and attains comparable result on the Count task. Our model requires less computation time and achieves better performance compared with the RNNs-based methods. Additional ablation study demonstrates the effect of each component of our proposed model.

A Novel Method of Visual Attention for Targets Detection

2013 ◽  
Vol 347-350 ◽  
pp. 3764-3768 ◽  
Author(s):  
Zhuo Zhang ◽  
Xin Nan Fan ◽  
Xue Wu Zhang ◽  
Hai Yan Xu ◽  
Min Li
Keyword(s):  
Visual Attention ◽  
Information Extraction ◽  
Feature Space ◽  
Gain Factor ◽  
Top Down ◽  
Bottom Up ◽  
Attention Model ◽  
Proposed Model ◽  
Novel Method ◽  
Statistical Prior

Inspired by the research of human visual system in neuroanatomy and psychology, the paper proposes a two-way collaborative visual attention model for target detection.In this new method , bottom-up attention information cooperates with top-down attention information to detect a target rapidly and accuractly. Firstly,the statistical prior knowledge of target and background is applied to optimize bottom-up attention information in different feature space and scale space.Secondly, after the SNR of salience difference between target and interference is computed ,the bottom-up gain factor is obtained.Thirdly, the gain factor is applied to adjust bottom up attention information extraction and then to maximize the salience contrast of target and background.Finally, target is detected by adjusted saliency.Experimental results shows that the proposed model in this paper can improve the real-time capability and reliability of target detection.

scholarly journals DeepStory: Video Story QA by Deep Embedded Memory Networks

2017 ◽  
Author(s):  
Kyung-Min Kim ◽  
Min-Oh Heo ◽  
Seong-Ho Choi ◽  
Byoung-Tak Zhang
Keyword(s):  
Question Answering ◽  
State Of The Art ◽  
Video Stream ◽  
Long Term Memory ◽  
Embedded Memory ◽  
Term Memory ◽  
Attention Model ◽  
Fine Grained ◽  
Scene Description

Question-answering (QA) on video contents is a significant challenge for achieving human-level intelligence as it involves both vision and language in real-world settings. Here we demonstrate the possibility of an AI agent performing video story QA by learning from a large amount of cartoon videos. We develop a video-story learning model, i.e. Deep Embedded Memory Networks (DEMN), to reconstruct stories from a joint scene-dialogue video stream using a latent embedding space of observed data. The video stories are stored in a long-term memory component. For a given question, an LSTM-based attention model uses the long-term memory to recall the best question-story-answer triplet by focusing on specific words containing key information. We trained the DEMN on a novel QA dataset of children’s cartoon video series, Pororo. The dataset contains 16,066 scene-dialogue pairs of 20.5-hour videos, 27,328 fine-grained sentences for scene description, and 8,913 story-related QA pairs. Our experimental results show that the DEMN outperforms other QA models. This is mainly due to 1) the reconstruction of video stories in a scene-dialogue combined form that utilize the latent embedding and 2) attention. DEMN also achieved state-of-the-art results on the MovieQA benchmark.

Sign in / Sign up

Export Citation Format

Share Document