scholarly journals Combating Fake News with Transformers: A Comparative Analysis of Stance Detection and Subjectivity Analysis

Information ◽  
2021 ◽  
Vol 12 (10) ◽  
pp. 409
Author(s):  
Panagiotis Kasnesis ◽  
Lazaros Toumanidis ◽  
Charalampos Z. Patrikakis
Keyword(s):  
Social Networks ◽  
Comparative Analysis ◽  
Language Processing ◽  
State Of The Art ◽  
Language Models ◽  
Modular Approach ◽  
Human Beings ◽  
Fake News ◽  
The Past ◽  
Subjectivity Analysis

The widespread use of social networks has brought to the foreground a very important issue, the veracity of the information circulating within them. Many natural language processing methods have been proposed in the past to assess a post’s content with respect to its reliability; however, end-to-end approaches are not comparable in ability to human beings. To overcome this, in this paper, we propose the use of a more modular approach that produces indicators about a post’s subjectivity and the stance provided by the replies it has received to date, letting the user decide whether (s)he trusts or does not trust the provided information. To this end, we fine-tuned state-of-the-art transformer-based language models and compared their performance with previous related work on stance detection and subjectivity analysis. Finally, we discuss the obtained results.

scholarly journals Comparative Analysis of Transformer based Language Models

2021 ◽  
Author(s):  
Aman Pathak
Keyword(s):  
Comparative Analysis ◽  
Natural Language ◽  
Language Processing ◽  
State Of The Art ◽  
Language Models ◽  
Learning Models ◽  
Contributing Factors ◽  
Language Abilities ◽  
The Past ◽  
Power Standard

Natural language processing (NLP) has witnessed many substantial advancements in the past three years. With the introduction of the Transformer and self-attention mechanism, language models are now able to learn better representations of the natural language. These attentionbased models have achieved exceptional state-of-the-art results on various NLP benchmarks. One of the contributing factors is the growing use of transfer learning. Models are pre-trained on unsupervised objectives using rich datasets that develop fundamental natural language abilities that are fine-tuned further on supervised data for downstream tasks. Surprisingly, current researches have led to a novel era of powerful models that no longer require finetuning. The objective of this paper is to present a comparative analysis of some of the most influential language models. The benchmarks of the study are problem-solving methodologies, model architecture, compute power, standard NLP benchmark accuracies and shortcomings.

Modeling Rumors in Twitter

2016 ◽  
Vol 3 (4) ◽  
pp. 46-67 ◽  
Author(s):  
Rhythm Walia ◽  
M.P.S. Bhatia
Keyword(s):  
Social Networks ◽  
Social Media ◽  
Comparative Analysis ◽  
State Of The Art ◽  
Viral Marketing ◽  
Daily Lives ◽  
The Past ◽  
Depth Analysis ◽  
Analysis Of Results ◽  
Effective Channel

With the advent of web 2.0 and anonymous free Internet services available to almost everyone, social media has gained immense popularity in disseminating information. It has become an effective channel for advertising and viral marketing. People rely on social networks for news, communication and it has become an integral part of our daily lives. But due to the limited accountability of users, it is often misused for the spread of rumors. Such rumor diffusion hampers the credibility of social media and may spread social panic. Analyzing rumors in social media has gained immense attention from the researchers in the past decade. In this paper the authors provide a survey of work in rumor analysis, which will serve as a stepping-stone for new researchers. They organized the study of rumors into four categories and discussed state of the art papers in each with an in-depth analysis of results of different models used and a comparative analysis between approaches used by different authors.

Towards corpus and model: Hierarchical structured-attention-based features for Indonesian named entity recognition

2021 ◽  
pp. 1-12
Author(s):  
Yingwen Fu ◽  
Nankai Lin ◽  
Xiaotian Lin ◽  
Shengyi Jiang
Keyword(s):  
Language Processing ◽  
State Of The Art ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Language Models ◽  
Neural Models ◽  
Performance Models ◽  
Named Entity ◽  
High Resource ◽  
Benchmark Datasets

Named entity recognition (NER) is fundamental to natural language processing (NLP). Most state-of-the-art researches on NER are based on pre-trained language models (PLMs) or classic neural models. However, these researches are mainly oriented to high-resource languages such as English. While for Indonesian, related resources (both in dataset and technology) are not yet well-developed. Besides, affix is an important word composition for Indonesian language, indicating the essentiality of character and token features for token-wise Indonesian NLP tasks. However, features extracted by currently top-performance models are insufficient. Aiming at Indonesian NER task, in this paper, we build an Indonesian NER dataset (IDNER) comprising over 50 thousand sentences (over 670 thousand tokens) to alleviate the shortage of labeled resources in Indonesian. Furthermore, we construct a hierarchical structured-attention-based model (HSA) for Indonesian NER to extract sequence features from different perspectives. Specifically, we use an enhanced convolutional structure as well as an enhanced attention structure to extract deeper features from characters and tokens. Experimental results show that HSA establishes competitive performance on IDNER and three benchmark datasets.

scholarly journals Improving Semantic Similarity with Cross-Lingual Resources: A Study in Bangla—A Low Resourced Language

Informatics ◽  
2019 ◽  
Vol 6 (2) ◽  
pp. 19 ◽  
Author(s):  
Rajat Pandit ◽  
Saptarshi Sengupta ◽  
Sudip Kumar Naskar ◽  
Niladri Sekhar Dash ◽  
Mohini Mohan Sardar
Keyword(s):  
Semantic Similarity ◽  
Language Processing ◽  
State Of The Art ◽  
Human Beings ◽  
Human Evaluation ◽  
Evaluation Scores ◽  
Similarity Calculation ◽  
Marked Enhancement ◽  
Cross Lingual ◽  
Standing Problem

Semantic similarity is a long-standing problem in natural language processing (NLP). It is a topic of great interest as its understanding can provide a look into how human beings comprehend meaning and make associations between words. However, when this problem is looked at from the viewpoint of machine understanding, particularly for under resourced languages, it poses a different problem altogether. In this paper, semantic similarity is explored in Bangla, a less resourced language. For ameliorating the situation in such languages, the most rudimentary method (path-based) and the latest state-of-the-art method (Word2Vec) for semantic similarity calculation were augmented using cross-lingual resources in English and the results obtained are truly astonishing. In the presented paper, two semantic similarity approaches have been explored in Bangla, namely the path-based and distributional model and their cross-lingual counterparts were synthesized in light of the English WordNet and Corpora. The proposed methods were evaluated on a dataset comprising of 162 Bangla word pairs, which were annotated by five expert raters. The correlation scores obtained between the four metrics and human evaluation scores demonstrate a marked enhancement that the cross-lingual approach brings into the process of semantic similarity calculation for Bangla.

Știrile false: limite și perspective ale analizei lingvistice

Transilvania ◽  
2020 ◽  
pp. 72-77
Author(s):  
Anca-Simina Martin ◽  
Simina-Maria Terian
Keyword(s):  
Critical Discourse Analysis ◽  
Language Processing ◽  
Quantitative Methods ◽  
Critical Discussion ◽  
Critical Discourse ◽  
Fake News ◽  
Proper Understanding ◽  
Qualitative And Quantitative Methods ◽  
The Past ◽  
Processing Techniques

This article sets out to offer an overview and a review of the latest linguistic research into fake news. To this end, the authors put forward a critical discussion of the paradigms and instruments deployed over the past decade to analyze and identify this textual (micro)genre, from natural language processing techniques to critical discourse analysis. The conclusion of our study is that a proper understanding of the fake news phenomenon can only be achieved by bringing together qualitative and quantitative methods.

scholarly journals Text: An R-package for Analyzing and Visualizing Human Language Using Natural Language Processing and Deep Learning

2021 ◽  
Author(s):  
Oscar Nils Erik Kjell ◽  
H. Andrew Schwartz ◽  
Salvatore Giorgi
Keyword(s):  
Deep Learning ◽  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Rating Scale ◽  
State Of The Art ◽  
R Package ◽  
Language Models ◽  
Categorical Variables ◽  
Human Language

The language that individuals use for expressing themselves contains rich psychological information. Recent significant advances in Natural Language Processing (NLP) and Deep Learning (DL), namely transformers, have resulted in large performance gains in tasks related to understanding natural language such as machine translation. However, these state-of-the-art methods have not yet been made easily accessible for psychology researchers, nor designed to be optimal for human-level analyses. This tutorial introduces text (www.r-text.org), a new R-package for analyzing and visualizing human language using transformers, the latest techniques from NLP and DL. Text is both a modular solution for accessing state-of-the-art language models and an end-to-end solution catered for human-level analyses. Hence, text provides user-friendly functions tailored to test hypotheses in social sciences for both relatively small and large datasets. This tutorial describes useful methods for analyzing text, providing functions with reliable defaults that can be used off-the-shelf as well as providing a framework for the advanced users to build on for novel techniques and analysis pipelines. The reader learns about six methods: 1) textEmbed: to transform text to traditional or modern transformer-based word embeddings (i.e., numeric representations of words); 2) textTrain: to examine the relationships between text and numeric/categorical variables; 3) textSimilarity and 4) textSimilarityTest: to computing semantic similarity scores between texts and significance test the difference in meaning between two sets of texts; and 5) textProjection and 6) textProjectionPlot: to examine and visualize text within the embedding space according to latent or specified construct dimensions (e.g., low to high rating scale scores).

Modeling Rumors in Twitter

2020 ◽  
pp. 192-215
Author(s):  
Rhythm Walia ◽  
M.P.S. Bhatia
Keyword(s):  
Social Networks ◽  
Social Media ◽  
Comparative Analysis ◽  
Web 2.0 ◽  
Viral Marketing ◽  
Daily Lives ◽  
The Past ◽  
Depth Analysis ◽  
Analysis Of Results ◽  
Effective Channel

With the advent of web 2.0 and anonymous free Internet services available to almost everyone, social media has gained immense popularity in disseminating information. It has become an effective channel for advertising and viral marketing. People rely on social networks for news, communication and it has become an integral part of our daily lives. But due to the limited accountability of users, it is often misused for the spread of rumors. Such rumor diffusion hampers the credibility of social media and may spread social panic. Analyzing rumors in social media has gained immense attention from the researchers in the past decade. In this paper the authors provide a survey of work in rumor analysis, which will serve as a stepping-stone for new researchers. They organized the study of rumors into four categories and discussed state of the art papers in each with an in-depth analysis of results of different models used and a comparative analysis between approaches used by different authors.

A Deep Learning Based Approach for Fake News Detection

2021 ◽  
pp. 388-394
Author(s):  
Srishti Sharma ◽  
Vaishali Kalra
Keyword(s):  
Social Networks ◽  
Social Media ◽  
Language Processing ◽  
Research Effort ◽  
Political Polarization ◽  
Fake News ◽  
User Characteristics ◽  
Mainstream Media ◽  
Boosting Method ◽  
Partisan Conflict

Owing to the rapid explosion of social media platforms in the past decade, we spread and consume information via the internet at an expeditious rate. It has caused an alarming proliferation of fake news on social networks. The global nature of social networks has facilitated international blowout of fake news. Fake news has proven to increase political polarization and partisan conflict. Fake news is also found to be more rampant on social media than mainstream media. The evil of fake news is garnering a lot of attention and research effort. In this work, we have tried to handle the spread of fake news via tweets. We have performed fake news classification by employing user characteristics as well as tweet text. Thus, trying to provide a holistic solution for fake news detection. For classifying user characteristics, we have used the XGBoost algorithm which is an ensemble of decision trees utilising the boosting method. Further to correctly classify the tweet text we used various natural language processing techniques to preprocess the tweets and then applied a sequential neural network and state-of-the-art BERT transformer to classify the tweets. The models have then been evaluated and compared with various baseline models to show that our approach effectively tackles this problemOwing to the rapid explosion of social media platforms in the past decade, we spread and consume information via the internet at an expeditious rate. It has caused an alarming proliferation of fake news on social networks. The global nature of social networks has facilitated international blowout of fake news. Fake news has proven to increase political polarization and partisan conflict. Fake news is also found to be more rampant on social media than mainstream media. The evil of fake news is garnering a lot of attention and research effort. In this work, we have tried to handle the spread of fake news via tweets. We have performed fake news classification by employing user characteristics as well as tweet text. Thus, trying to provide a holistic solution for fake news detection. For classifying user characteristics, we have used the XGBoost algorithm which is an ensemble of decision trees utilising the boosting method. Further to correctly classify the tweet text we used various natural language processing techniques to preprocess the tweets and then applied a sequential neural network and state-of-the-art BERT transformer to classify the tweets. The models have then been evaluated and compared with various baseline models to show that our approach effectively tackles this problem

scholarly journals Concept Recognition as a Machine Translation Problem

2020 ◽  
Author(s):  
Mayla R Boguslav ◽  
Negacy D Hailu ◽  
Michael Bada ◽  
William A Baumgartner ◽  
Lawrence E Hunter
Keyword(s):  
Machine Learning ◽  
Machine Translation ◽  
Language Processing ◽  
State Of The Art ◽  
Training Data ◽  
Language Models ◽  
Alternative Methods ◽  
Automated Assignment ◽  
Concept Recognition ◽  
Alternative Approaches

AbstractBackgroundAutomated assignment of specific ontology concepts to mentions in text is a critical task in biomedical natural language processing, and the subject of many open shared tasks. Although the current state of the art involves the use of neural network language models as a post-processing step, the very large number of ontology classes to be recognized and the limited amount of gold-standard training data has impeded the creation of end-to-end systems based entirely on machine learning. Recently, Hailu et al. recast the concept recognition problem as a type of machine translation and demonstrated that sequence-to-sequence machine learning models had the potential to outperform multi-class classification approaches. Here we systematically characterize the factors that contribute to the accuracy and efficiency of several approaches to sequence-to-sequence machine learning.ResultsWe report on our extensive studies of alternative methods and hyperparameter selections. The results not only identify the best-performing systems and parameters across a wide variety of ontologies but also illuminate about the widely varying resource requirements and hyperparameter robustness of alternative approaches. Analysis of the strengths and weaknesses of such systems suggest promising avenues for future improvements as well as design choices that can increase computational efficiency with small costs in performance. Bidirectional Encoder Representations from Transformers for Biomedical Text Mining (BioBERT) for span detection (as previously found) along with the Open-source Toolkit for Neural Machine Translation (OpenNMT) for concept normalization achieve state-of-the-art performance for most ontologies in CRAFT Corpus. This approach uses substantially fewer computational resources, including hardware, memory, and time than several alternative approaches.ConclusionsMachine translation is a promising avenue for fully machine-learning-based concept recognition that achieves state-of-the-art results on the CRAFT Corpus, evaluated via a direct comparison to previous results from the 2019 CRAFT Shared Task. Experiments illuminating the reasons for the surprisingly good performance of sequence-to-sequence methods targeting ontology identifiers suggest that further progress may be possible by mapping to alternative target concept representations. All code and models can be found at: https://github.com/UCDenver-ccp/Concept-Recognition-as-Translation.

scholarly journals Transformer protein language models are unsupervised structure learners

2020 ◽  
Author(s):  
Roshan M Rao ◽  
Joshua Meier ◽  
Tom Sercu ◽  
Sergey Ovchinnikov ◽  
Alexander Rives
Keyword(s):  
Protein Structure ◽  
State Of The Art ◽  
Protein Structure Determination ◽  
Language Modeling ◽  
Language Models ◽  
Evolutionary Constraints ◽  
Functional Constraints ◽  
Contact Prediction ◽  
The Past ◽  
Potential Alternative

Unsupervised contact prediction is central to uncovering physical, structural, and functional constraints for protein structure determination and design. For decades, the predominant approach has been to infer evolutionary constraints from a set of related sequences. In the past year, protein language models have emerged as a potential alternative, but performance has fallen short of state-of-the-art approaches in bioinformatics. In this paper we demonstrate that Transformer attention maps learn contacts from the unsupervised language modeling objective. We find the highest capacity models that have been trained to date already outperform a state-of-the-art unsupervised contact prediction pipeline, suggesting these pipelines can be replaced with a single forward pass of an end-to-end model.

Sign in / Sign up

Export Citation Format

Share Document