scholarly journals Matrix Completion for Graph-Based Deep Semi-Supervised Learning

2019 ◽  
Vol 33 ◽  
pp. 5058-5065 ◽  
Author(s):  
Fariborz Taherkhani ◽  
Hadi Kazemi ◽  
Nasser M. Nasrabadi
Keyword(s):  
Supervised Learning ◽  
Matrix Completion ◽  
Unlabeled Data ◽  
Classification Problems ◽  
Class Label ◽  
Data Points ◽  
Small Set ◽  
Similarity Graph ◽  
Deep Learning Model ◽  
The Web

Convolutional Neural Networks (CNNs) have provided promising achievements for image classification problems. However, training a CNN model relies on a large number of labeled data. Considering the vast amount of unlabeled data available on the web, it is important to make use of these data in conjunction with a small set of labeled data to train a deep learning model. In this paper, we introduce a new iterative Graph-based Semi-Supervised Learning (GSSL) method to train a CNN-based classifier using a large amount of unlabeled data and a small amount of labeled data. In this method, we first construct a similarity graph in which the nodes represent the CNN features corresponding to data points (labeled and unlabeled) while the edges tend to connect the data points with the same class label. In this graph, the missing label of unsupervised nodes is predicted by using a matrix completion method based on rank minimization criterion. In the next step, we use the constructed graph to calculate triplet regularization loss which is added to the supervised loss obtained by initially labeled data to update the CNN network parameters.

Semi-Supervised Learning

2011 ◽  
pp. 1022-1027
Author(s):  
Tobias Scheffer
Keyword(s):  
Supervised Learning ◽  
Supervised Classification ◽  
Unlabeled Data ◽  
Training Data ◽  
Classification Algorithms ◽  
Classification Problems

For many classification problems, unlabeled training data are inexpensive and readily available, whereas labeling training data imposes costs. Semi-supervised classification algorithms aim at utilizing information contained in unlabeled data in addition to the (few) labeled data.

scholarly journals On Teacher-Student Semi-Supervised Learning for Chest X-ray Image Classification

2021 ◽  
Author(s):  
Roberto Augusto Philippi Martins ◽  
Danilo Silva
Keyword(s):  
Deep Learning ◽  
Supervised Learning ◽  
Learning Algorithm ◽  
Unlabeled Data ◽  
Imaging Data ◽  
X Ray ◽  
Teacher Student ◽  
Chest X Ray ◽  
Deep Learning Model

The lack of labeled data is one of the main prohibiting issues on the development of deep learning models, as they rely on large labeled datasets in order to achieve high accuracy in complex tasks. Our objective is to evaluate the performance gain of having additional unlabeled data in the training of a deep learning model when working with medical imaging data. We present a semi-supervised learning algorithm that utilizes a teacher-student paradigm in order to leverage unlabeled data in the classification of chest X-ray images. Using our algorithm on the ChestX-ray14 dataset, we manage to achieve a substantial increase in performance when using small labeled datasets. With our method, a model achieves an AUROC of 0.822 with only 2% labeled data and 0.865 with 5% labeled data, while a fully supervised method achieves an AUROC of 0.807 with 5% labeled data and only 0.845 with 10%.

scholarly journals Semi-Supervised Learning for Maximizing the Partial AUC

2020 ◽  
Vol 34 (04) ◽  
pp. 4239-4246
Author(s):  
Tomoharu Iwata ◽  
Akinori Fujino ◽  
Naonori Ueda
Keyword(s):  
Supervised Learning ◽  
Binary Classification ◽  
Characteristic Curve ◽  
False Positive Rate ◽  
True Positive Rate ◽  
Unlabeled Data ◽  
Spam Filtering ◽  
Classification Problems ◽  
Partial Area ◽  
Positive Rate

The partial area under a receiver operating characteristic curve (pAUC) is a performance measurement for binary classification problems that summarizes the true positive rate with the specific range of the false positive rate. Obtaining classifiers that achieve high pAUC is important in a wide variety of applications, such as cancer screening and spam filtering. Although many methods have been proposed for maximizing the pAUC, existing methods require many labeled data for training. In this paper, we propose a semi-supervised learning method for maximizing the pAUC, which trains a classifier with a small amount of labeled data and a large amount of unlabeled data. To exploit the unlabeled data, we derive two approximations of the pAUC: the first is calculated from positive and unlabeled data, and the second is calculated from negative and unlabeled data. A classifier is trained by maximizing the weighted sum of the two approximations of the pAUC and the pAUC that is calculated from positive and negative data. With experiments using various datasets, we demonstrate that the proposed method achieves higher test pAUCs than existing methods.

scholarly journals DEVELOPMENT AND COMPARATIVE ANALYSIS OF SEMI-SUPERVISED LEARNING ALGORITHMS ON A SMALL AMOUNT OF LABELED DATA

2021 ◽  
pp. 98-103
Author(s):  
Klym Yamkovyi
Keyword(s):  
Supervised Learning ◽  
Nearest Neighbor ◽  
Learning Algorithm ◽  
Center Of Mass ◽  
Unlabeled Data ◽  
Learning Approaches ◽  
Classification Problems ◽  
K Nearest Neighbor ◽  
Supervised Learning Algorithms ◽  
Label Information

The paper is dedicated to the development and comparative experimental analysis of semi-supervised learning approaches based on a mix of unsupervised and supervised approaches for the classification of datasets with a small amount of labeled data, namely, identifying to which of a set of categories a new observation belongs using a training set of data containing observations whose category membership is known. Semi-supervised learning is an approach to machine learning that combines a small amount of labeled data with a large amount of unlabeled data during training. Unlabeled data, when used in combination with a small quantity of labeled data, can produce significant improvement in learning accuracy. The goal is semi-supervised methods development and analysis along with comparing their accuracy and robustness on different synthetics datasets. The proposed approach is based on the unsupervised K-medoids methods, also known as the Partitioning Around Medoid algorithm, however, unlike Kmedoids the proposed algorithm first calculates medoids using only labeled data and next process unlabeled classes – assign labels of nearest medoid. Another proposed approach is the mix of the supervised method of K-nearest neighbor and unsupervised K-Means. Thus, the proposed learning algorithm uses information about both the nearest points and classes centers of mass. The methods have been implemented using Python programming language and experimentally investigated for solving classification problems using datasets with different distribution and spatial characteristics. Datasets were generated using the scikit-learn library. Was compared the developed approaches to find average accuracy on all these datasets. It was shown, that even small amounts of labeled data allow us to use semi-supervised learning, and proposed modifications ensure to improve accuracy and algorithm performance, which was demonstrated during experiments. And with the increase of available label information accuracy of the algorithms grows up. Thus, the developed algorithms are using a distance metric that considers available label information. Keywords: Unsupervised learning, supervised learning. semi-supervised learning, clustering, distance, distance function, nearest neighbor, medoid, center of mass.

BATCH MODE ACTIVE LEARNING FOR GRAPH-BASED SEMI-SUPERVISED LEARNING

2013 ◽  
Vol 27 (07) ◽  
pp. 1351002
Author(s):  
CHEONG HEE PARK
Keyword(s):  
Active Learning ◽  
Supervised Learning ◽  
Selection Process ◽  
Unlabeled Data ◽  
Large Set ◽  
Class Label ◽  
Batch Mode ◽  
Beneficial Effects ◽  
Label Information ◽  
Improved Performance

In semi-supervised learning, when the number of data samples with class label information is very small, information from unlabeled data is utilized in the learning process. Many semi-supervised learning methods have been presented and have exhibited competent performance. Active learning also aims to overcome the shortage of labeled data by obtaining class labels for some selected unlabeled data from experts. However, the selection process for the most informative unlabeled data samples can be demanding when the search is performed over a large set of unlabeled data. In this paper, we propose a method for batch mode active learning in graph-based semi-supervised learning. Instead of acquiring class label information of one unlabeled data sample at a time, we obtain information about several data samples at once, reducing time complexity while preserving the beneficial effects of active learning. Experimental results demonstrate the improved performance of the proposed method.

Semi-Supervised Learning

2011 ◽  
pp. 1787-1793
Author(s):  
Tobias Scheffer
Keyword(s):  
Active Learning ◽  
Supervised Learning ◽  
Error Rate ◽  
Learning Algorithms ◽  
Gaussian Mixture ◽  
Unlabeled Data ◽  
Training Data ◽  
Support Vector ◽  
Classification Problems ◽  
Class Labels

For many classification problems, unlabeled training data are inexpensive and readily available, whereas labeling training data imposes costs. Semi-supervised classification algorithms aim at utilizing information contained in unlabeled data in addition to the (few) labeled data. Semi-supervised (for an example, see Seeger, 2001) has a long tradition in statistics (Cooper & Freeman, 1970); much early work has focused on Bayesian discrimination of Gaussians. The Expectation Maximization (EM) algorithm (Dempster, Laird, & Rubin, 1977) is the most popular method for learning generative models from labeled and unlabeled data. Model-based, generative learning algorithms find model parameters (e.g., the parameters of a Gaussian mixture model) that best explain the available labeled and unlabeled data, and they derive the discriminating classification hypothesis from this model. In discriminative learning, unlabeled data is typically incorporated via the integration of some model assumption into the discriminative framework (Miller & Uyar, 1997; Titterington, Smith, & Makov, 1985). The Transductive Support Vector Machine (Vapnik, 1998; Joachims, 1999) uses unlabeled data to identify a hyperplane that has a large distance not only from the labeled data but also from all unlabeled data. This identification results in a bias toward placing the hyperplane in regions of low density p(x). Recently, studies have covered graph-based approaches that rely on the assumption that neighboring instances are more likely to belong to the same class than remote instances (Blum & Chawla, 2001). A distinct approach to utilizing unlabeled data has been proposed by de Sa (1994), Yarowsky (1995) and Blum and Mitchell (1998). When the available attributes can be split into independent and compatible subsets, then multi-view learning algorithms can be employed. Multi-view algorithms, such as co-training (Blum & Mitchell, 1998) and co-EM (Nigam & Ghani, 2000), learn two independent hypotheses, which bootstrap by providing each other with labels for the unlabeled data. An analysis of why training two independent hypotheses that provide each other with conjectured class labels for unlabeled data might be better than EM-like self-training has been provided by Dasgupta, Littman, and McAllester (2001) and has been simplified by Abney (2002). The disagreement rate of two independent hypotheses is an upper bound on the error rate of either hypothesis. Multi-view algorithms minimize the disagreement rate between the peer hypotheses (a situation that is most apparent for the algorithm of Collins & Singer, 1999) and thereby the error rate. Semi-supervised learning is related to active learning. Active learning algorithms are able to actively query the class labels of unlabeled data. By contrast, semi-supervised algorithms are bound to learn from the given data.

scholarly journals Combining active learning and graph-based semi-supervised learning

2019 ◽  
Author(s):  
Jhonatan Candao ◽  
Lilian Berton
Keyword(s):  
Active Learning ◽  
Supervised Learning ◽  
Unlabeled Data ◽  
Experimental Results ◽  
Huge Amount ◽  
Label Data ◽  
Small Set

The scarcity of labeled data is a common problem in many applications. Semi-supervised learning (SSL) aims to minimize the need for human annotation combining a small set of label data with a huge amount of unlabeled data. Similarly to SSL, Active Learning (AL) reduces the annotation efforts selecting the most informative points for annotation. Few works explore AL and graph-based SSL, in this work, we combine both strategies and explore different techniques: two graph-based SSL and two query strategy of AL in a pool-based scenario. Experimental results in artificial and real datasets indicate that our approach requires significantly less labeled instances to reach the same performance of random label selection.

scholarly journals Learning From Labeled And Unlabeled Data: An Empirical Study Across Techniques And Domains

2005 ◽  
Vol 23 ◽  
pp. 331-366 ◽  
Author(s):  
N. V. Chawla ◽  
G. Karakoulas
Keyword(s):  
Empirical Study ◽  
Supervised Learning ◽  
Selection Bias ◽  
Sample Selection ◽  
Poor Performance ◽  
Unlabeled Data ◽  
Learning Techniques ◽  
Different Types ◽  
Data Points ◽  
The Impact

There has been increased interest in devising learning techniques that combine unlabeled data with labeled data ? i.e. semi-supervised learning. However, to the best of our knowledge, no study has been performed across various techniques and different types and amounts of labeled and unlabeled data. Moreover, most of the published work on semi-supervised learning techniques assumes that the labeled and unlabeled data come from the same distribution. It is possible for the labeling process to be associated with a selection bias such that the distributions of data points in the labeled and unlabeled sets are different. Not correcting for such bias can result in biased function approximation with potentially poor performance. In this paper, we present an empirical study of various semi-supervised learning techniques on a variety of datasets. We attempt to answer various questions such as the effect of independence or relevance amongst features, the effect of the size of the labeled and unlabeled sets and the effect of noise. We also investigate the impact of sample-selection bias on the semi-supervised learning techniques under study and implement a bivariate probit technique particularly designed to correct for such bias.

Semi-Supervised Classification by Particle Competition in Complex Network’s Edges

2016 ◽  
Vol 30 (09) ◽  
pp. 1660006
Author(s):  
Paulo Roberto Urio ◽  
Filipe Alves Neto Verri ◽  
Liang Zhao
Keyword(s):  
Supervised Learning ◽  
Complex Network ◽  
Computer Simulations ◽  
Supervised Classification ◽  
Model Simulation ◽  
Unlabeled Data ◽  
Biologically Inspired ◽  
Learning Tasks ◽  
Input Dataset ◽  
Data Points

We present a biologically inspired model for transductive semi-supervised learning tasks. Specifically, this model consists of a set of particles that walk and compete in a complex network. From an input dataset, similarities between labeled and unlabeled data points derive a network representation. As particles walk the network, they compete to dominate the edges. Over the process, particles can become inactive, and, to compensate, labeled vertices will feed new particles to the system. Resulted from the model simulation, we analyze sets of edges arranged by their label dominance. Each set forms a subnetwork that is used to classify connected vertices. Our computer simulations on artificial and real datasets show that this technique can classify nonlinearly formed data and detect vertices of different classes in overlapping regions.

scholarly journals Training Complex Models with Multi-Task Weak Supervision

2019 ◽  
Vol 33 ◽  
pp. 4763-4771 ◽  
Author(s):  
Alexander Ratner ◽  
Braden Hancock ◽  
Jared Dunnmon ◽  
Frederic Sala ◽  
Shreyash Pandey ◽  
...  
Keyword(s):  
Majority Vote ◽  
Matrix Completion ◽  
Classification Problems ◽  
Weak Supervision ◽  
Dependency Structure ◽  
Fine Grained ◽  
Data Points ◽  
Complex Models ◽  
Dependency Structures ◽  
Different Levels

As machine learning models continue to increase in complexity, collecting large hand-labeled training sets has become one of the biggest roadblocks in practice. Instead, weaker forms of supervision that provide noisier but cheaper labels are often used. However, these weak supervision sources have diverse and unknown accuracies, may output correlated labels, and may label different tasks or apply at different levels of granularity. We propose a framework for integrating and modeling such weak supervision sources by viewing them as labeling different related sub-tasks of a problem, which we refer to as the multi-task weak supervision setting. We show that by solving a matrix completion-style problem, we can recover the accuracies of these multi-task sources given their dependency structure, but without any labeled data, leading to higher-quality supervision for training an end model. Theoretically, we show that the generalization error of models trained with this approach improves with the number of unlabeled data points, and characterize the scaling with respect to the task and dependency structures. On three fine-grained classification problems, we show that our approach leads to average gains of 20.2 points in accuracy over a traditional supervised approach, 6.8 points over a majority vote baseline, and 4.1 points over a previously proposed weak supervision method that models tasks separately.

Sign in / Sign up

Export Citation Format

Share Document