scholarly journals Dual Adversarial Co-Learning for Multi-Domain Text Classification

2020 ◽  
Vol 34 (04) ◽  
pp. 6438-6445
Author(s):  
Yuan Wu ◽  
Yuhong Guo
Keyword(s):  
Text Classification ◽  
State Of The Art ◽  
Digital Data ◽  
Classification Model ◽  
Classification Models ◽  
Learning Framework ◽  
Good Classification ◽  
Classification Tasks ◽  
Multiple Domains ◽  
Learned Features

With the advent of deep learning, the performance of text classification models have been improved significantly. Nevertheless, the successful training of a good classification model requires a sufficient amount of labeled data, while it is always expensive and time consuming to annotate data. With the rapid growth of digital data, similar classification tasks can typically occur in multiple domains, while the availability of labeled data can largely vary across domains. Some domains may have abundant labeled data, while in some other domains there may only exist a limited amount (or none) of labeled data. Meanwhile text classification tasks are highly domain-dependent — a text classifier trained in one domain may not perform well in another domain. In order to address these issues, in this paper we propose a novel dual adversarial co-learning approach for multi-domain text classification (MDTC). The approach learns shared-private networks for feature extraction and deploys dual adversarial regularizations to align features across different domains and between labeled and unlabeled data simultaneously under a discrepancy based co-learning framework, aiming to improve the classifiers' generalization capacity with the learned features. We conduct experiments on multi-domain sentiment classification datasets. The results show the proposed approach achieves the state-of-the-art MDTC performance.

scholarly journals Research on Real-Time Multiple Single Garbage Classification Based on Convolutional Neural Network

2020 ◽  
Vol 2020 ◽  
pp. 1-6
Author(s):  
Jian-ye Yuan ◽  
Xin-yuan Nan ◽  
Cheng-rong Li ◽  
Le-le Sun
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Real Time ◽  
Environmental Science ◽  
Fine Tuning ◽  
Classification Model ◽  
Classification Models ◽  
Data Set ◽  
Classification Tasks ◽  
Accuracy Rates

Considering that the garbage classification is urgent, a 23-layer convolutional neural network (CNN) model is designed in this paper, with the emphasis on the real-time garbage classification, to solve the low accuracy of garbage classification and recycling and difficulty in manual recycling. Firstly, the depthwise separable convolution was used to reduce the Params of the model. Then, the attention mechanism was used to improve the accuracy of the garbage classification model. Finally, the model fine-tuning method was used to further improve the performance of the garbage classification model. Besides, we compared the model with classic image classification models including AlexNet, VGG16, and ResNet18 and lightweight classification models including MobileNetV2 and SuffleNetV2 and found that the model GAF_dense has a higher accuracy rate, fewer Params, and FLOPs. To further check the performance of the model, we tested the CIFAR-10 data set and found the accuracy rates of the model (GAF_dense) are 0.018 and 0.03 higher than ResNet18 and SufflenetV2, respectively. In the ImageNet data set, the accuracy rates of the model (GAF_dense) are 0.225 and 0.146 higher than Resnet18 and SufflenetV2, respectively. Therefore, the garbage classification model proposed in this paper is suitable for garbage classification and other classification tasks to protect the ecological environment, which can be applied to classification tasks such as environmental science, children’s education, and environmental protection.

scholarly journals Enhancing the performance of cancer text classification model based on cancer hallmarks

2021 ◽  
Vol 10 (2) ◽  
pp. 316
Author(s):  
Noha Ali ◽  
Ahmed H. AbuEl-Atta ◽  
Hala H. Zayed
Keyword(s):  
Neural Network ◽  
Language Processing ◽  
Text Classification ◽  
State Of The Art ◽  
Classification Model ◽  
Biomedical Text ◽  
Cancer Hallmarks ◽  
Embedding Technique ◽  
Proposed Model ◽  
Biomedical Text Classification

<span id="docs-internal-guid-cb130a3a-7fff-3e11-ae3d-ad2310e265f8"><span>Deep learning (DL) algorithms achieved state-of-the-art performance in computer vision, speech recognition, and natural language processing (NLP). In this paper, we enhance the convolutional neural network (CNN) algorithm to classify cancer articles according to cancer hallmarks. The model implements a recent word embedding technique in the embedding layer. This technique uses the concept of distributed phrase representation and multi-word phrases embedding. The proposed model enhances the performance of the existing model used for biomedical text classification. The result of the proposed model overcomes the previous model by achieving an F-score equal to 83.87% using an unsupervised technique that trained on PubMed abstracts called PMC vectors (PMCVec) embedding. Also, we made another experiment on the same dataset using the recurrent neural network (RNN) algorithm with two different word embeddings Google news and PMCVec which achieving F-score equal to 74.9% and 76.26%, respectively.</span></span>

Improving Techniques for Naïve Bayes Text Classifiers

2010 ◽  
pp. 111-127
Author(s):  
Han-joon Kim
Keyword(s):  
Text Classification ◽  
Naive Bayes ◽  
Naïve Bayes ◽  
Classification Systems ◽  
Classification Model ◽  
Learning Approaches ◽  
Learning Framework ◽  
The Em Algorithm ◽  
Meta Learning ◽  
Text Classifiers

This chapter introduces two practical techniques for improving Naïve Bayes text classifiers that are widely used for text classification. The Naïve Bayes has been evaluated to be a practical text classification algorithm due to its simple classification model, reasonable classification accuracy, and easy update of classification model. Thus, many researchers have a strong incentive to improve the Naïve Bayes by combining it with other meta-learning approaches such as EM (Expectation Maximization) and Boosting. The EM approach is to combine the Naïve Bayes with the EM algorithm and the Boosting approach is to use the Naïve Bayes as a base classifier in the AdaBoost algorithm. For both approaches, a special uncertainty measure fit for Naïve Bayes learning is used. In the Naïve Bayes learning framework, these approaches are expected to be practical solutions to the problem of lack of training documents in text classification systems.

scholarly journals Graph Debiased Contrastive Learning with Joint Representation Clustering

2021 ◽  
Author(s):  
Han Zhao ◽  
Xu Yang ◽  
Zhenru Wang ◽  
Erkun Yang ◽  
Cheng Deng
Keyword(s):  
State Of The Art ◽  
False Negative ◽  
Poor Performance ◽  
Representation Learning ◽  
Graph Representation ◽  
Learning Framework ◽  
Clustering And Classification ◽  
Class Information ◽  
Classification Tasks ◽  
Joint Representation

By contrasting positive-negative counterparts, graph contrastive learning has become a prominent technique for unsupervised graph representation learning. However, existing methods fail to consider the class information and will introduce false-negative samples in the random negative sampling, causing poor performance. To this end, we propose a graph debiased contrastive learning framework, which can jointly perform representation learning and clustering. Specifically, representations can be optimized by aligning with clustered class information, and simultaneously, the optimized representations can promote clustering, leading to more powerful representations and clustering results. More importantly, we randomly select negative samples from the clusters which are different from the positive sample's cluster. In this way, as the supervisory signals, the clustering results can be utilized to effectively decrease the false-negative samples. Extensive experiments on five datasets demonstrate that our method achieves new state-of-the-art results on graph clustering and classification tasks.

scholarly journals Completely Heterogeneous Transfer Learning with Attention - What And What Not To Transfer

2017 ◽  
Author(s):  
Seungwhan Moon ◽  
Jaime Carbonell
Keyword(s):  
Transfer Learning ◽  
Text Classification ◽  
State Of The Art ◽  
A Priori ◽  
Classification Task ◽  
Learning Approach ◽  
Learning Framework ◽  
Previous State

We study a transfer learning framework where source and target datasets are heterogeneous in both feature and label spaces. Specifically, we do not assume explicit relations between source and target tasks a priori, and thus it is crucial to determine what and what not to transfer from source knowledge. Towards this goal, we define a new heterogeneous transfer learning approach that (1) selects and attends to an optimized subset of source samples to transfer knowledge from, and (2) builds a unified transfer network that learns from both source and target knowledge. This method, termed "Attentional Heterogeneous Transfer", along with a newly proposed unsupervised transfer loss, improve upon the previous state-of-the-art approaches on extensive simulations as well as a challenging hetero-lingual text classification task.

scholarly journals Attention enhanced capsule network for text classification by encoding syntactic dependency trees with graph convolutional neural network

2022 ◽  
Vol 7 ◽  
pp. e831
Author(s):  
Xudong Jia ◽  
Li Wang
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Text Classification ◽  
Classification Model ◽  
Network Graph ◽  
Word Sequence ◽  
Dependency Trees ◽  
Classification Tasks ◽  
Topic Labeling ◽  
Syntactic Dependency

Text classification is a fundamental task in many applications such as topic labeling, sentiment analysis, and spam detection. The text syntactic relationship and word sequence are important and useful for text classification. How to model and incorporate them to improve performance is one key challenge. Inspired by human behavior in understanding text. In this paper, we combine the syntactic relationship, sequence structure, and semantics for text representation, and propose an attention-enhanced capsule network-based text classification model. Specifically, we use graph convolutional neural networks to encode syntactic dependency trees, build multi-head attention to encode dependencies relationship in text sequence, merge with semantic information by capsule network at last. Extensive experiments on five datasets demonstrate that our approach can effectively improve the performance of text classification compared with state-of-the-art methods. The result also shows capsule network, graph convolutional neural network, and multi-headed attention has integration effects on text classification tasks.

scholarly journals Margin-Based Pareto Ensemble Pruning: An Ensemble Pruning Algorithm That Learns to Search Optimized Ensembles

2019 ◽  
Vol 2019 ◽  
pp. 1-12 ◽  
Author(s):  
Ruihan Hu ◽  
Songbin Zhou ◽  
Yisen Liu ◽  
Zhiri Tang
Keyword(s):  
Machine Learning ◽  
State Of The Art ◽  
Classification Performance ◽  
Test Set ◽  
Pruning Algorithm ◽  
Ensemble Pruning ◽  
Learning Framework ◽  
Classification Tasks ◽  
Validation Set ◽  
Definition Of

The ensemble pruning system is an effective machine learning framework that combines several learners as experts to classify a test set. Generally, ensemble pruning systems aim to define a region of competence based on the validation set to select the most competent ensembles from the ensemble pool with respect to the test set. However, the size of the ensemble pool is usually fixed, and the performance of an ensemble pool heavily depends on the definition of the region of competence. In this paper, a dynamic pruning framework called margin-based Pareto ensemble pruning is proposed for ensemble pruning systems. The framework explores the optimized ensemble pool size during the overproduction stage and finetunes the experts during the pruning stage. The Pareto optimization algorithm is used to explore the size of the overproduction ensemble pool that can result in better performance. Considering the information entropy of the learners in the indecision region, the marginal criterion for each learner in the ensemble pool is calculated using margin criterion pruning, which prunes the experts with respect to the test set. The effectiveness of the proposed method for classification tasks is assessed using datasets. The results show that margin-based Pareto ensemble pruning can achieve smaller ensemble sizes and better classification performance in most datasets when compared with state-of-the-art models.

scholarly journals Text classification models for the automatic detection of nonmedical prescription medication use from social media

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Mohammed Ali Al-Garadi ◽  
Yuan-Chi Yang ◽  
Haitao Cai ◽  
Yucheng Ruan ◽  
Karen O’Connor ◽  
...  
Keyword(s):  
Machine Learning ◽  
Social Media ◽  
Deep Learning ◽  
Text Classification ◽  
Prescription Medication ◽  
The United States ◽  
Language Models ◽  
Classification Model ◽  
Future Research ◽  
Classification Models

Abstract Background Prescription medication (PM) misuse/abuse has emerged as a national crisis in the United States, and social media has been suggested as a potential resource for performing active monitoring. However, automating a social media-based monitoring system is challenging—requiring advanced natural language processing (NLP) and machine learning methods. In this paper, we describe the development and evaluation of automatic text classification models for detecting self-reports of PM abuse from Twitter. Methods We experimented with state-of-the-art bi-directional transformer-based language models, which utilize tweet-level representations that enable transfer learning (e.g., BERT, RoBERTa, XLNet, AlBERT, and DistilBERT), proposed fusion-based approaches, and compared the developed models with several traditional machine learning, including deep learning, approaches. Using a public dataset, we evaluated the performances of the classifiers on their abilities to classify the non-majority “abuse/misuse” class. Results Our proposed fusion-based model performs significantly better than the best traditional model (F1-score [95% CI]: 0.67 [0.64–0.69] vs. 0.45 [0.42–0.48]). We illustrate, via experimentation using varying training set sizes, that the transformer-based models are more stable and require less annotated data compared to the other models. The significant improvements achieved by our best-performing classification model over past approaches makes it suitable for automated continuous monitoring of nonmedical PM use from Twitter. Conclusions BERT, BERT-like and fusion-based models outperform traditional machine learning and deep learning models, achieving substantial improvements over many years of past research on the topic of prescription medication misuse/abuse classification from social media, which had been shown to be a complex task due to the unique ways in which information about nonmedical use is presented. Several challenges associated with the lack of context and the nature of social media language need to be overcome to further improve BERT and BERT-like models. These experimental driven challenges are represented as potential future research directions.

scholarly journals Do Not Have Enough Data? Deep Learning to the Rescue!

2020 ◽  
Vol 34 (05) ◽  
pp. 7383-7390 ◽  
Author(s):  
Ateret Anaby-Tavor ◽  
Boaz Carmeli ◽  
Esther Goldbraich ◽  
Amir Kantor ◽  
George Kour ◽  
...  
Keyword(s):  
Text Classification ◽  
Data Augmentation ◽  
State Of The Art ◽  
Language Model ◽  
Original Data ◽  
Fine Tuning ◽  
Initial Training ◽  
Series Of Experiments ◽  
Classification Tasks ◽  
Trained Neural Network

Based on recent advances in natural language modeling and those in text generation capabilities, we propose a novel data augmentation method for text classification tasks. We use a powerful pre-trained neural network model to artificially synthesize new labeled data for supervised learning. We mainly focus on cases with scarce labeled data. Our method, referred to as language-model-based data augmentation (LAMBADA), involves fine-tuning a state-of-the-art language generator to a specific task through an initial training phase on the existing (usually small) labeled data. Using the fine-tuned model and given a class label, new sentences for the class are generated. Our process then filters these new sentences by using a classifier trained on the original data. In a series of experiments, we show that LAMBADA improves classifiers' performance on a variety of datasets. Moreover, LAMBADA significantly improves upon the state-of-the-art techniques for data augmentation, specifically those applicable to text classification tasks with little data.

scholarly journals Text Classification Models for the Automatic Detection of Nonmedical Prescription Medication Use from Social Media

2020 ◽  
Author(s):  
Ali Al-Garadi Mohammed ◽  
Yuan-Chi Yang ◽  
Haitao Cai ◽  
Yucheng Ruan ◽  
Karen O’Connor ◽  
...  
Keyword(s):  
Machine Learning ◽  
Social Media ◽  
Language Processing ◽  
Text Classification ◽  
Prescription Medication ◽  
The United States ◽  
Language Models ◽  
Classification Model ◽  
Classification Models ◽  
Active Monitoring

ABSTRACTPrescription medication (PM) misuse/abuse has emerged as a national crisis in the United States, and social media has been suggested as a potential resource for performing active monitoring. However, automating a social media-based monitoring system is challenging—requiring advanced natural language processing (NLP) and machine learning methods. In this paper, we describe the development and evaluation of automatic text classification models for detecting self-reports of PM abuse from Twitter. We experimented with state-of-the-art bi-directional transformer-based language models, which utilize tweet-level representations that enable transfer learning (e.g., BERT, RoBERTa, XLNet, AlBERT, and DistilBERT), proposed fusion-based approaches, and compared the developed models with several traditional machine learning, including deep learning, approaches. Using a public dataset, we evaluated the performances of the classifiers on their abilities to classify the non-majority “abuse/misuse” class. Our proposed fusion-based model performs significantly better than the best traditional model (F1-score [95% CI]: 0.67 [0.64-0.69] vs. 0.45 [0.42-0.48]). We illustrate, via experimentation using differing training set sizes, that the transformer-based models are more stable and require less annotated data compared to the other models. The significant improvements achieved by our best-performing classification model over past approaches makes it suitable for automated continuous monitoring of nonmedical PM use from Twitter.

Sign in / Sign up

Export Citation Format

Share Document