scholarly journals Active Learning with Query Generation for Cost-Effective Text Classification

2020 ◽  
Vol 34 (04) ◽  
pp. 6583-6590
Author(s):  
Yi-Fan Yan ◽  
Sheng-Jun Huang ◽  
Shaoyi Chen ◽  
Meng Liao ◽  
Jin Xu
Keyword(s):  
Active Learning ◽  
Text Classification ◽  
Large Scale ◽  
Cost Effective ◽  
Classification Performance ◽  
Unlabeled Data ◽  
Classification Model ◽  
Class Label ◽  
Text Document ◽  
Query Generation

Labeling a text document is usually time consuming because it requires the annotator to read the whole document and check its relevance with each possible class label. It thus becomes rather expensive to train an effective model for text classification when it involves a large dataset of long documents. In this paper, we propose an active learning approach for text classification with lower annotation cost. Instead of scanning all the examples in the unlabeled data pool to select the best one for query, the proposed method automatically generates the most informative examples based on the classification model, and thus can be applied to tasks with large scale or even infinite unlabeled data. Furthermore, we propose to approximate the generated example with a few summary words by sparse reconstruction, which allows the annotators to easily assign the class label by reading a few words rather than the long document. Experiments on different datasets demonstrate that the proposed approach can effectively improve the classification performance while significantly reduce the annotation cost.

scholarly journals Text Classification of Public Feedbacks using Convolutional Neural Network Based on Differential Evolution Algorithm

2019 ◽  
Vol 14 (1) ◽  
pp. 124-134 ◽  
Author(s):  
Shuai Zhang ◽  
Yong Chen ◽  
Xiaoling Huang ◽  
Yishuai Cai
Keyword(s):  
Neural Network ◽  
Differential Evolution ◽  
Convolutional Neural Network ◽  
Text Classification ◽  
Differential Evolution Algorithm ◽  
Classification Performance ◽  
Classification Model ◽  
Evolution Algorithm ◽  
Classification Prediction

Online feedback is an effective way of communication between government departments and citizens. However, the daily high number of public feedbacks has increased the burden on government administrators. The deep learning method is good at automatically analyzing and extracting deep features of data, and then improving the accuracy of classification prediction. In this study, we aim to use the text classification model to achieve the automatic classification of public feedbacks to reduce the work pressure of administrator. In particular, a convolutional neural network model combined with word embedding and optimized by differential evolution algorithm is adopted. At the same time, we compared it with seven common text classification models, and the results show that the model we explored has good classification performance under different evaluation metrics, including accuracy, precision, recall, and F1-score.

scholarly journals Classification of Imbalanced Cloud Image Data Using Deep Neural Networks –Performance Improvement Through a Data Science Competition–

2021 ◽  
Author(s):  
Daisuke Matsuoka
Keyword(s):  
Neural Networks ◽  
Data Science ◽  
Image Data ◽  
Cost Effective ◽  
Classification Performance ◽  
Classification Model ◽  
Extreme Weather Events ◽  
Target Class ◽  
Imbalanced Classification ◽  
Classification Prediction

Abstract Image data classification using machine learning is one of the effective methods for detecting atmospheric phenomena. However, extreme weather events with a small number of cases cause a decrease in classification prediction accuracy owing to the imbalance of data between the target class and the other classes. In order to build a highly accurate classification model, we held a data analysis competition to determine the best classification performance for two classes of cloud image data: tropical cyclones including precursors and other classes. For the top models in the competition, minority data oversampling, majority data undersampling, ensemble learning, deep layer neural networks, and cost-effective loss functions were used to improve the imbalanced classification performance. In particular, the best model out of 209 submissions succeeded in improving the classification capability by 65.4% over similar conventional methods in a measure of low false alarm ratio.

scholarly journals Deep Active Learning for Classifying Cancer Pathology Reports

2020 ◽  
Author(s):  
Kevin De Angeli ◽  
Shang Gao ◽  
Mohammed Alawad ◽  
Hong-Jun Yoon ◽  
Noah Schaefferkoetter ◽  
...  
Keyword(s):  
Active Learning ◽  
Learning Strategies ◽  
Text Classification ◽  
Random Sampling ◽  
Learning Strategy ◽  
Classification Model ◽  
Cancer Pathology ◽  
Learning Techniques ◽  
Active Learning Strategies ◽  
Pathology Reports

Abstract Background: Automated text classification has many important applications in the clinical setting; however, obtaining labelled data for training machine learning and deep learning models is often difficult and expensive. Active learning techniques may mitigate this challenge by reducing the amount of labelled data required to effectively train a model. In this study, we analyze the effectiveness of eleven active learning algorithms on classifying subsite and histology from cancer pathology reports using a Convolutional Neural Network (CNN) as the text classification model. Results: We compare the performance of each active learning strategy using two differently sized datasets and two different classification tasks. Our results show that on all tasks and dataset sizes, all active learning strategies except diversity-sampling strategies outperformed random sampling, i.e., no active learning. On our large dataset (15K initial labelled samples, adding 15K additional labelled samples each iteration of active learning), there was no clear winner between the different active learning strategies. On our small dataset (1K initial labelled samples, adding 1K additional labelled samples each iteration of active learning), marginal and ratio uncertainty sampling performed better than all other active learning techniques. We found that compared to random sampling, active learning strongly helps performance on rare classes by focusing on underrepresented classes. Conclusions: Active learning can save annotation cost by helping human annotators efficiently and intelligently select which samples to label. Our results show that a dataset constructed using effective active learning techniques requires less than half the amount of labelled data to achieve the same performance as a dataset that constructed using random sampling.

scholarly journals Cost-Effective Active Learning from Diverse Labelers

2017 ◽  
Author(s):  
Sheng-Jun Huang ◽  
Jia-Lve Chen ◽  
Xin Mu ◽  
Zhi-Hua Zhou
Keyword(s):  
Cost Effectiveness ◽  
Active Learning ◽  
Low Cost ◽  
Selection Criterion ◽  
Ground Truth ◽  
Cost Effective ◽  
Classification Model ◽  
Data Sets ◽  
Active Selection ◽  
The Cost

In traditional active learning, there is only one labeler that always returns the ground truth of queried labels. However, in many applications, multiple labelers are available to offer diverse qualities of labeling with different costs. In this paper, we perform active selection on both instances and labelers, aiming to improve the classification model most with the lowest cost. While the cost of a labeler is proportional to its overall labeling quality, we also observe that different labelers usually have diverse expertise, and thus it is likely that labelers with a low overall quality can provide accurate labels on some specific instances. Based on this fact, we propose a novel active selection criterion to evaluate the cost-effectiveness of instance-labeler pairs, which ensures that the selected instance is helpful for improving the classification model, and meanwhile the selected labeler can provide an accurate label for the instance with a relative low cost. Experiments on both UCI and real crowdsourcing data sets demonstrate the superiority of our proposed approach on selecting cost-effective queries.

Leveraging Unlabeled Data for Classification

2011 ◽  
pp. 1164-1169
Author(s):  
Yinghui Yang ◽  
Balaji Padmanabhan
Keyword(s):  
Research Question ◽  
Unlabeled Data ◽  
Training Data ◽  
Bank Loan ◽  
Classification Model ◽  
Classification Models ◽  
Class Label ◽  
Data Record ◽  
Model Training ◽  
Class Labels

Classification is a form of data analysis that can be used to extract models to predict categorical class labels (Han & Kamber, 2001). Data classification has proven to be very useful in a wide variety of applications. For example, a classification model can be built to categorize bank loan applications as either safe or risky. In order to build a classification model, training data containing multiple independent variables and a dependant variable (class label) is needed. If a data record has a known value for its class label, this data record is termed “labeled”. If the value for its class is unknown, it is “unlabeled”. There are situations with a large amount of unlabeled data and a small amount of labeled data. Using only labeled data to build classification models can potentially ignore useful information contained in the unlabeled data. Furthermore, unlabeled data can often be much cheaper and more plentiful than labeled data, and so if useful information can be extracted from it that reduces the need for labeled examples, this can be a significant benefit (Balcan & Blum 2005). The default practice is to use only the labeled data to build a classification model and then assign class labels to the unlabeled data. However, when the amount of labeled data is not enough, the classification model built only using the labeled data can be biased and far from accurate. The class labels assigned to the unlabeled data can then be inaccurate. How to leverage the information contained in the unlabeled data to help improve the accuracy of the classification model is an important research question. There are two streams of research that addresses the challenging issue of how to appropriately use unlabeled data for building classification models. The details are discussed below.

scholarly journals Co-Training Semi-Supervised Deep Learning for Sentiment Classification of MOOC Forum Posts

Symmetry ◽  
2019 ◽  
Vol 12 (1) ◽  
pp. 8
Author(s):  
Jing Chen ◽  
Jun Feng ◽  
Xia Sun ◽  
Yang Liu
Keyword(s):  
Deep Learning ◽  
Large Scale ◽  
Sample Selection ◽  
Classification Performance ◽  
Sentiment Classification ◽  
Unlabeled Data ◽  
Learning Performance ◽  
Average Accuracy ◽  
Complex Features

Sentiment classification of forum posts of massive open online courses is essential for educators to make interventions and for instructors to improve learning performance. Lacking monitoring on learners’ sentiments may lead to high dropout rates of courses. Recently, deep learning has emerged as an outstanding machine learning technique for sentiment classification, which extracts complex features automatically with rich representation capabilities. However, deep neural networks always rely on a large amount of labeled data for supervised training. Constructing large-scale labeled training datasets for sentiment classification is very laborious and time consuming. To address this problem, this paper proposes a co-training, semi-supervised deep learning model for sentiment classification, leveraging limited labeled data and massive unlabeled data simultaneously to achieve performance comparable to those methods trained on massive labeled data. To satisfy the condition of two views of co-training, we encoded texts into vectors from views of word embedding and character-based embedding independently, considering words’ external and internal information. To promote the classification performance with limited data, we propose a double-check strategy sample selection method to select samples with high confidence to augment the training set iteratively. In addition, we propose a mixed loss function both considering the labeled data with asymmetric and unlabeled data. Our proposed method achieved a 89.73% average accuracy and an 93.55% average F1-score, about 2.77% and 3.2% higher than baseline methods. Experimental results demonstrate the effectiveness of the proposed model trained on limited labeled data, which performs much better than those trained on massive labeled data.

BATCH MODE ACTIVE LEARNING FOR GRAPH-BASED SEMI-SUPERVISED LEARNING

2013 ◽  
Vol 27 (07) ◽  
pp. 1351002
Author(s):  
CHEONG HEE PARK
Keyword(s):  
Active Learning ◽  
Supervised Learning ◽  
Selection Process ◽  
Unlabeled Data ◽  
Large Set ◽  
Class Label ◽  
Batch Mode ◽  
Beneficial Effects ◽  
Label Information ◽  
Improved Performance

In semi-supervised learning, when the number of data samples with class label information is very small, information from unlabeled data is utilized in the learning process. Many semi-supervised learning methods have been presented and have exhibited competent performance. Active learning also aims to overcome the shortage of labeled data by obtaining class labels for some selected unlabeled data from experts. However, the selection process for the most informative unlabeled data samples can be demanding when the search is performed over a large set of unlabeled data. In this paper, we propose a method for batch mode active learning in graph-based semi-supervised learning. Instead of acquiring class label information of one unlabeled data sample at a time, we obtain information about several data samples at once, reducing time complexity while preserving the beneficial effects of active learning. Experimental results demonstrate the improved performance of the proposed method.

Improved Fake Reviews Detection Model Based on Vertical Ensemble Tri-Training and Active Learning

2021 ◽  
Vol 12 (3) ◽  
pp. 1-19
Author(s):  
Chunyong Yin ◽  
Haoqi Cuan ◽  
Yuhang Zhu ◽  
Zhichao Yin
Keyword(s):  
Active Learning ◽  
Large Scale ◽  
User Behavior ◽  
Classification Performance ◽  
Online Activity ◽  
Detection Model ◽  
Model Based ◽  
Group Integration ◽  
Good Classification Performance ◽  
Fake Reviews

People’s increasingly frequent online activity has generated a large number of reviews, whereas fake reviews can mislead users and harm their personal interests. In addition, it is not feasible to label reviews on a large scale because of the high cost of manual labeling. Therefore, to improve the detection performance by utilizing the unlabeled reviews, this article proposes a fake reviews detection model based on vertical ensemble tri-training and active learning (VETT-AL). The model combines the features of review text with the user behavior features as feature extraction. In the VETT-AL algorithm, the iterative process is divided into two parts: vertical integration within the group and horizontal integration among the groups. The intra-group integration is to integrate three original classifiers by using the previous iterative models of the classifiers. The inter-group integration is to adopt the active learning based on entropy to select the data with the highest confidence and label it, and as the result of that, the second generation classifiers are trained by the traditional process to improve the accuracy of the label. Experimental results show that the proposed model has a good classification performance.

scholarly journals Active Learning Strategies for Phenotypic Profiling of High-Content Screens

2014 ◽  
Vol 19 (5) ◽  
pp. 685-695 ◽  
Author(s):  
Kevin Smith ◽  
Peter Horvath
Keyword(s):  
Active Learning ◽  
Learning Strategies ◽  
Large Scale ◽  
Classification Performance ◽  
New Drugs ◽  
Supervised Machine Learning ◽  
Biological Research ◽  
Time Investment ◽  
Active Learning Strategies ◽  
The Impact

High-content screening is a powerful method to discover new drugs and carry out basic biological research. Increasingly, high-content screens have come to rely on supervised machine learning (SML) to perform automatic phenotypic classification as an essential step of the analysis. However, this comes at a cost, namely, the labeled examples required to train the predictive model. Classification performance increases with the number of labeled examples, and because labeling examples demands time from an expert, the training process represents a significant time investment. Active learning strategies attempt to overcome this bottleneck by presenting the most relevant examples to the annotator, thereby achieving high accuracy while minimizing the cost of obtaining labeled data. In this article, we investigate the impact of active learning on single-cell–based phenotype recognition, using data from three large-scale RNA interference high-content screens representing diverse phenotypic profiling problems. We consider several combinations of active learning strategies and popular SML methods. Our results show that active learning significantly reduces the time cost and can be used to reveal the same phenotypic targets identified using SML. We also identify combinations of active learning strategies and SML methods which perform better than others on the phenotypic profiling problems we studied.

Sign in / Sign up

Export Citation Format

Share Document