Adversarial attacks on text classification models using layer‐wise relevance propagation

2020 ◽  
Vol 35 (9) ◽  
pp. 1397-1415
Author(s):  
Jincheng Xu ◽  
Qingfeng Du
Keyword(s):  
Text Classification ◽  
Classification Models

scholarly journals GILE: A Generalized Input-Label Embedding for Text Classification

2019 ◽  
Vol 7 ◽  
pp. 139-155 ◽  
Author(s):  
Nikolaos Pappas ◽  
James Henderson
Keyword(s):  
Text Classification ◽  
Joint Space ◽  
Classification Performance ◽  
Cross Entropy ◽  
Categorical Variables ◽  
Classification Models ◽  
Set Size ◽  
Nonlinear Input ◽  
Label Semantics

Neural text classification models typically treat output labels as categorical variables that lack description and semantics. This forces their parametrization to be dependent on the label set size, and, hence, they are unable to scale to large label sets and generalize to unseen ones. Existing joint input-label text models overcome these issues by exploiting label descriptions, but they are unable to capture complex label relationships, have rigid parametrization, and their gains on unseen labels happen often at the expense of weak performance on the labels seen during training. In this paper, we propose a new input-label model that generalizes over previous such models, addresses their limitations, and does not compromise performance on seen labels. The model consists of a joint nonlinear input-label embedding with controllable capacity and a joint-space-dependent classification unit that is trained with cross-entropy loss to optimize classification performance. We evaluate models on full-resource and low- or zero-resource text classification of multilingual news and biomedical text with a large label set. Our model outperforms monolingual and multilingual models that do not leverage label semantics and previous joint input-label space models in both scenarios.

scholarly journals Identification of Judicial Outcomes in Judgments: A Generalized Gini-PLS Approach

Stats ◽  
2020 ◽  
Vol 3 (4) ◽  
pp. 427-443
Author(s):  
Gildas Tagny-Ngompé ◽  
Stéphane Mussard ◽  
Guillaume Zambrano ◽  
Sébastien Harispe ◽  
Jacky Montmain
Keyword(s):  
Law Enforcement ◽  
Success Rate ◽  
Text Classification ◽  
Binary Classification ◽  
Classification Algorithms ◽  
Classification Models ◽  
Target Variable ◽  
Technical Aspects ◽  
Legal Documents ◽  
Judicial Outcomes

This paper presents and compares several text classification models that can be used to extract the outcome of a judgment from justice decisions, i.e., legal documents summarizing the different rulings made by a judge. Such models can be used to gather important statistics about cases, e.g., success rate based on specific characteristics of cases’ parties or jurisdiction, and are therefore important for the development of Judicial prediction not to mention the study of Law enforcement in general. We propose in particular the generalized Gini-PLS which better considers the information in the distribution tails while attenuating, as in the simple Gini-PLS, the influence exerted by outliers. Modeling the studied task as a supervised binary classification, we also introduce the LOGIT-Gini-PLS suited to the explanation of a binary target variable. In addition, various technical aspects regarding the evaluated text classification approaches which consists of combinations of representations of judgments and classification algorithms are studied using an annotated corpora of French justice decisions.

scholarly journals A Survey of Arabic Text Classification Models

2018 ◽  
Vol 8 (6) ◽  
pp. 4352 ◽  
Author(s):  
Ahed M. F. Al-Sbou
Keyword(s):  
Language Processing ◽  
Text Classification ◽  
Arabic Language ◽  
Arabic Text ◽  
Classification Models ◽  
Natural Languages ◽  
Text Organization ◽  
Arabic Text Classification ◽  
Arabic Language Processing

<p>There is a huge content of Arabic text available over online that requires an organization of these texts. As result, here are many applications of natural languages processing (NLP) that concerns with text organization. One of the is text classification (TC). TC helps to make dealing with unorganized text. However, it is easier to classify them into suitable class or labels. This paper is a survey of Arabic text classification. Also, it presents comparison among different methods in the classification of Arabic texts, where Arabic text is represented a complex text due to its vocabularies. Arabic language is one of the richest languages in the world, where it has many linguistic bases. The researche in Arabic language processing is very few compared to English. As a result, these problems represent challenges in the classification, and organization of specific Arabic text. Text classification (TC) helps to access the most documents, or information that has already classified into specific classes, or categories to one or more classes or categories. In addition, classification of documents facilitate search engine to decrease the amount of document to, and then to become easier to search and matching with queries.</p>

scholarly journals Dual Adversarial Co-Learning for Multi-Domain Text Classification

2020 ◽  
Vol 34 (04) ◽  
pp. 6438-6445
Author(s):  
Yuan Wu ◽  
Yuhong Guo
Keyword(s):  
Text Classification ◽  
State Of The Art ◽  
Digital Data ◽  
Classification Model ◽  
Classification Models ◽  
Learning Framework ◽  
Good Classification ◽  
Classification Tasks ◽  
Multiple Domains ◽  
Learned Features

With the advent of deep learning, the performance of text classification models have been improved significantly. Nevertheless, the successful training of a good classification model requires a sufficient amount of labeled data, while it is always expensive and time consuming to annotate data. With the rapid growth of digital data, similar classification tasks can typically occur in multiple domains, while the availability of labeled data can largely vary across domains. Some domains may have abundant labeled data, while in some other domains there may only exist a limited amount (or none) of labeled data. Meanwhile text classification tasks are highly domain-dependent — a text classifier trained in one domain may not perform well in another domain. In order to address these issues, in this paper we propose a novel dual adversarial co-learning approach for multi-domain text classification (MDTC). The approach learns shared-private networks for feature extraction and deploys dual adversarial regularizations to align features across different domains and between labeled and unlabeled data simultaneously under a discrepancy based co-learning framework, aiming to improve the classifiers' generalization capacity with the learned features. We conduct experiments on multi-domain sentiment classification datasets. The results show the proposed approach achieves the state-of-the-art MDTC performance.

Towards Combining Object Detection and Text Classification Models for Form Entity Recognition

2021 ◽  
pp. 196-205
Author(s):  
María Villota ◽  
Gonzalo Santamaría ◽  
César Domínguez ◽  
Jónathan Heras ◽  
Eloy Mata ◽  
...  
Keyword(s):  
Object Detection ◽  
Text Classification ◽  
Entity Recognition ◽  
Classification Models

scholarly journals Text classification models for the automatic detection of nonmedical prescription medication use from social media

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Mohammed Ali Al-Garadi ◽  
Yuan-Chi Yang ◽  
Haitao Cai ◽  
Yucheng Ruan ◽  
Karen O’Connor ◽  
...  
Keyword(s):  
Machine Learning ◽  
Social Media ◽  
Deep Learning ◽  
Text Classification ◽  
Prescription Medication ◽  
The United States ◽  
Language Models ◽  
Classification Model ◽  
Future Research ◽  
Classification Models

Abstract Background Prescription medication (PM) misuse/abuse has emerged as a national crisis in the United States, and social media has been suggested as a potential resource for performing active monitoring. However, automating a social media-based monitoring system is challenging—requiring advanced natural language processing (NLP) and machine learning methods. In this paper, we describe the development and evaluation of automatic text classification models for detecting self-reports of PM abuse from Twitter. Methods We experimented with state-of-the-art bi-directional transformer-based language models, which utilize tweet-level representations that enable transfer learning (e.g., BERT, RoBERTa, XLNet, AlBERT, and DistilBERT), proposed fusion-based approaches, and compared the developed models with several traditional machine learning, including deep learning, approaches. Using a public dataset, we evaluated the performances of the classifiers on their abilities to classify the non-majority “abuse/misuse” class. Results Our proposed fusion-based model performs significantly better than the best traditional model (F1-score [95% CI]: 0.67 [0.64–0.69] vs. 0.45 [0.42–0.48]). We illustrate, via experimentation using varying training set sizes, that the transformer-based models are more stable and require less annotated data compared to the other models. The significant improvements achieved by our best-performing classification model over past approaches makes it suitable for automated continuous monitoring of nonmedical PM use from Twitter. Conclusions BERT, BERT-like and fusion-based models outperform traditional machine learning and deep learning models, achieving substantial improvements over many years of past research on the topic of prescription medication misuse/abuse classification from social media, which had been shown to be a complex task due to the unique ways in which information about nonmedical use is presented. Several challenges associated with the lack of context and the nature of social media language need to be overcome to further improve BERT and BERT-like models. These experimental driven challenges are represented as potential future research directions.

Sign in / Sign up

Export Citation Format

Share Document