scholarly journals Tag N’ Train: a technique to train improved classifiers on unlabeled data

2021 ◽  
Vol 2021 (1) ◽  
Author(s):  
Oz Amram ◽  
Cristina Mantilla Suarez
Keyword(s):  
Real Data ◽  
Unlabeled Data ◽  
Machine Learning Techniques ◽  
Jet Physics ◽  
Classification Problems ◽  
Weak Classifier ◽  
Potential Applications ◽  
Substantial Progress ◽  
Resonance Search ◽  
Class Labels

Abstract There has been substantial progress in applying machine learning techniques to classification problems in collider and jet physics. But as these techniques grow in sophistication, they are becoming more sensitive to subtle features of jets that may not be well modeled in simulation. Therefore, relying on simulations for training will lead to sub-optimal performance in data, but the lack of true class labels makes it difficult to train on real data. To address this challenge we introduce a new approach, called Tag N’ Train (TNT), that can be applied to unlabeled data that has two distinct sub-objects. The technique uses a weak classifier for one of the objects to tag signal-rich and background-rich samples. These samples are then used to train a stronger classifier for the other object. We demonstrate the power of this method by applying it to a dijet resonance search. By starting with autoencoders trained directly on data as the weak classifiers, we use TNT to train substantially improved classifiers. We show that Tag N’ Train can be a powerful tool in model-agnostic searches and discuss other potential applications.

Semi-Supervised Learning

2011 ◽  
pp. 1787-1793
Author(s):  
Tobias Scheffer
Keyword(s):  
Active Learning ◽  
Supervised Learning ◽  
Error Rate ◽  
Learning Algorithms ◽  
Gaussian Mixture ◽  
Unlabeled Data ◽  
Training Data ◽  
Support Vector ◽  
Classification Problems ◽  
Class Labels

For many classification problems, unlabeled training data are inexpensive and readily available, whereas labeling training data imposes costs. Semi-supervised classification algorithms aim at utilizing information contained in unlabeled data in addition to the (few) labeled data. Semi-supervised (for an example, see Seeger, 2001) has a long tradition in statistics (Cooper & Freeman, 1970); much early work has focused on Bayesian discrimination of Gaussians. The Expectation Maximization (EM) algorithm (Dempster, Laird, & Rubin, 1977) is the most popular method for learning generative models from labeled and unlabeled data. Model-based, generative learning algorithms find model parameters (e.g., the parameters of a Gaussian mixture model) that best explain the available labeled and unlabeled data, and they derive the discriminating classification hypothesis from this model. In discriminative learning, unlabeled data is typically incorporated via the integration of some model assumption into the discriminative framework (Miller & Uyar, 1997; Titterington, Smith, & Makov, 1985). The Transductive Support Vector Machine (Vapnik, 1998; Joachims, 1999) uses unlabeled data to identify a hyperplane that has a large distance not only from the labeled data but also from all unlabeled data. This identification results in a bias toward placing the hyperplane in regions of low density p(x). Recently, studies have covered graph-based approaches that rely on the assumption that neighboring instances are more likely to belong to the same class than remote instances (Blum & Chawla, 2001). A distinct approach to utilizing unlabeled data has been proposed by de Sa (1994), Yarowsky (1995) and Blum and Mitchell (1998). When the available attributes can be split into independent and compatible subsets, then multi-view learning algorithms can be employed. Multi-view algorithms, such as co-training (Blum & Mitchell, 1998) and co-EM (Nigam & Ghani, 2000), learn two independent hypotheses, which bootstrap by providing each other with labels for the unlabeled data. An analysis of why training two independent hypotheses that provide each other with conjectured class labels for unlabeled data might be better than EM-like self-training has been provided by Dasgupta, Littman, and McAllester (2001) and has been simplified by Abney (2002). The disagreement rate of two independent hypotheses is an upper bound on the error rate of either hypothesis. Multi-view algorithms minimize the disagreement rate between the peer hypotheses (a situation that is most apparent for the algorithm of Collins & Singer, 1999) and thereby the error rate. Semi-supervised learning is related to active learning. Active learning algorithms are able to actively query the class labels of unlabeled data. By contrast, semi-supervised algorithms are bound to learn from the given data.

Improved well-log classification using semisupervised label propagation and self-training, with comparisons to popular supervised algorithms

Geophysics ◽  
2019 ◽  
Vol 85 (1) ◽  
pp. O1-O15 ◽  
Author(s):  
Michael W. Dunham ◽  
Alison Malcolm ◽  
J. Kim Welford
Keyword(s):  
Unlabeled Data ◽  
Label Propagation ◽  
Machine Learning Techniques ◽  
Data Sets ◽  
Well Log ◽  
Training Method ◽  
Classification Problems ◽  
Data Set ◽  
Log Data ◽  
Supervised Methods

Machine-learning techniques allow geoscientists to extract meaningful information from data in an automated fashion, and they are also an efficient alternative to traditional manual interpretation methods. Many geophysical problems have an abundance of unlabeled data and a paucity of labeled data, and the lithology classification of wireline data reflects this situation. Training supervised algorithms on small labeled data sets can lead to overtraining, and subsequent predictions for the numerous unlabeled data may be unstable. However, semisupervised algorithms are designed for classification problems with limited amounts of labeled data, and they are theoretically able to achieve better accuracies than supervised algorithms in these situations. We explore this hypothesis by applying two semisupervised techniques, label propagation (LP) and self-training, to a well-log data set and compare their performance to three popular supervised algorithms. LP is an established method, but our self-training method is a unique adaptation of existing implementations. The well-log data were made public through an SEG competition held in 2016. We simulate a semisupervised scenario with these data by assuming that only one of the 10 wells has labels (i.e., core samples), and our objective is to predict the labels for the remaining nine wells. We generate results from these data in two stages. The first stage is applying all the algorithms in question to the data as is (i.e., the global data), and the results from this motivate the second stage, which is applying all algorithms to the data when they are decomposed into two separate data sets. Overall, our findings suggest that LP does not outperform the supervised methods, but our self-training method coupled with LP can outperform the supervised methods by a notable margin if the assumptions of LP are met.

scholarly journals Noise detection in classification problems

2017 ◽  
Author(s):  
Luís P. F. Garcia ◽  
Ana C. Lorena ◽  
André C. P. L. F. De Carvalho
Keyword(s):  
Machine Learning ◽  
Data Quality ◽  
Recommendation System ◽  
Predictive Performance ◽  
Real Data ◽  
Machine Learning Techniques ◽  
Noise Detection ◽  
Classification Problems ◽  
Learning Techniques ◽  
Meta Learning

Large volumes of data have been produced in many application domains. Nonetheless, when data quality is low, the performance of Machine Learning techniques is harmed. Real data are frequently affected by the presence of noise, which, when used in the training of Machine Learning techniques for predictive tasks, can result in complex models, with high induction time and low predictive performance. Identification and removal of noise can improve data quality and, as a result, the induced model. This thesis proposes new techniques for noise detection and the development of a recommendation system based on meta-learning to recommend the most suitable filter for new tasks. Experiments using artificial and real datasets show the relevance of this research.

A Comparative Study of Different Machine Learning Algorithms for Disease Prediction

2017 ◽  
Vol 7 (7) ◽  
pp. 172
Author(s):  
Anantvir Singh Romana
Keyword(s):  
Machine Learning ◽  
Subsequent Treatment ◽  
Machine Learning Algorithms ◽  
Machine Learning Techniques ◽  
Support Vector ◽  
Disease Prediction ◽  
Classification Problems ◽  
Learning Techniques ◽  
Neural Network Classifiers ◽  
Diagnostic Detection

Accurate diagnostic detection of the disease in a patient is critical and may alter the subsequent treatment and increase the chances of survival rate. Machine learning techniques have been instrumental in disease detection and are currently being used in various classification problems due to their accurate prediction performance. Various techniques may provide different desired accuracies and it is therefore imperative to use the most suitable method which provides the best desired results. This research seeks to provide comparative analysis of Support Vector Machine, Naïve bayes, J48 Decision Tree and neural network classifiers breast cancer and diabetes datsets.

scholarly journals Real UAV-Bird Image Classification Using CNN with a Synthetic Dataset

2021 ◽  
Vol 11 (9) ◽  
pp. 3863
Author(s):  
Ali Emre Öztürk ◽  
Ergun Erçelebi
Keyword(s):  
Deep Learning ◽  
Image Classification ◽  
Synthetic Data ◽  
Real Data ◽  
Corner Detection ◽  
Batch Size ◽  
Test Accuracy ◽  
Classification Problems ◽  
Auc Value ◽  
Classification Test

A large amount of training image data is required for solving image classification problems using deep learning (DL) networks. In this study, we aimed to train DL networks with synthetic images generated by using a game engine and determine the effects of the networks on performance when solving real-image classification problems. The study presents the results of using corner detection and nearest three-point selection (CDNTS) layers to classify bird and rotary-wing unmanned aerial vehicle (RW-UAV) images, provides a comprehensive comparison of two different experimental setups, and emphasizes the significant improvements in the performance in deep learning-based networks due to the inclusion of a CDNTS layer. Experiment 1 corresponds to training the commonly used deep learning-based networks with synthetic data and an image classification test on real data. Experiment 2 corresponds to training the CDNTS layer and commonly used deep learning-based networks with synthetic data and an image classification test on real data. In experiment 1, the best area under the curve (AUC) value for the image classification test accuracy was measured as 72%. In experiment 2, using the CDNTS layer, the AUC value for the image classification test accuracy was measured as 88.9%. A total of 432 different combinations of trainings were investigated in the experimental setups. The experiments were trained with various DL networks using four different optimizers by considering all combinations of batch size, learning rate, and dropout hyperparameters. The test accuracy AUC values for networks in experiment 1 ranged from 55% to 74%, whereas the test accuracy AUC values in experiment 2 networks with a CDNTS layer ranged from 76% to 89.9%. It was observed that the CDNTS layer has considerable effects on the image classification accuracy performance of deep learning-based networks. AUC, F-score, and test accuracy measures were used to validate the success of the networks.

scholarly journals Kernel Based Data-Adaptive Support Vector Machines for Multi-Class Classification

Mathematics ◽  
2021 ◽  
Vol 9 (9) ◽  
pp. 936
Author(s):  
Jianli Shao ◽  
Xin Liu ◽  
Wenqing He
Keyword(s):  
Machine Learning ◽  
Spatial Association ◽  
Class Imbalance ◽  
Imbalanced Data ◽  
Real Data ◽  
Kernel Functions ◽  
Support Vector ◽  
Classification Problems ◽  
Rare Class ◽  
Data Adaptive

Imbalanced data exist in many classification problems. The classification of imbalanced data has remarkable challenges in machine learning. The support vector machine (SVM) and its variants are popularly used in machine learning among different classifiers thanks to their flexibility and interpretability. However, the performance of SVMs is impacted when the data are imbalanced, which is a typical data structure in the multi-category classification problem. In this paper, we employ the data-adaptive SVM with scaled kernel functions to classify instances for a multi-class population. We propose a multi-class data-dependent kernel function for the SVM by considering class imbalance and the spatial association among instances so that the classification accuracy is enhanced. Simulation studies demonstrate the superb performance of the proposed method, and a real multi-class prostate cancer image dataset is employed as an illustration. Not only does the proposed method outperform the competitor methods in terms of the commonly used accuracy measures such as the F-score and G-means, but also successfully detects more than 60% of instances from the rare class in the real data, while the competitors can only detect less than 20% of the rare class instances. The proposed method will benefit other scientific research fields, such as multiple region boundary detection.

Global and Local Clustering-Based Regression Models to Forecast Power Consumption in Buildings

2016 ◽  
pp. 207-234
Author(s):  
Gonzalo Vergara ◽  
Juan J. Carrasco ◽  
Jesus Martínez-Gómez ◽  
Manuel Domínguez ◽  
José A. Gámez ◽  
...  
Keyword(s):  
Machine Learning ◽  
Power Consumption ◽  
Electric Power ◽  
Principal Component ◽  
Real Data ◽  
Machine Learning Techniques ◽  
Electric Power Consumption ◽  
Support Vector ◽  
Learning Machines ◽  
The University

The study of energy efficiency in buildings is an active field of research. Modeling and predicting energy related magnitudes leads to analyze electric power consumption and can achieve economical benefits. In this study, classical time series analysis and machine learning techniques, introducing clustering in some models, are applied to predict active power in buildings. The real data acquired corresponds to time, environmental and electrical data of 30 buildings belonging to the University of León (Spain). Firstly, we segmented buildings in terms of their energy consumption using principal component analysis. Afterwards, we applied state of the art machine learning methods and compare between them. Finally, we predicted daily electric power consumption profiles and compare them with actual data for different buildings. Our analysis shows that multilayer perceptrons have the lowest error followed by support vector regression and clustered extreme learning machines. We also analyze daily load profiles on weekdays and weekends for different buildings.

Machine Learning Applications in Mega-Text Processing

2010 ◽  
pp. 325-347 ◽  
Author(s):  
Marina Sokolova ◽  
Stan Szpakowicz
Keyword(s):  
Machine Learning ◽  
Language Processing ◽  
Opinion Mining ◽  
Personal Information ◽  
Text Processing ◽  
Machine Learning Techniques ◽  
Classification Problems ◽  
The Status ◽  
Machine Learning Applications ◽  
Applications Of Machine Learning

This chapter presents applications of machine learning techniques to problems in natural language processing that require work with very large amounts of text. Such problems came into focus after the Internet and other computer-based environments acquired the status of the prime medium for text delivery and exchange. In all cases which the authors discuss, an algorithm has ensured a meaningful result, be it the knowledge of consumer opinions, the protection of personal information or the selection of news reports. The chapter covers elements of opinion mining, news monitoring and privacy protection, and, in parallel, discusses text representation, feature selection, and word category and text classification problems. The applications presented here combine scientific interest and significant economic potential.

scholarly journals Learning from Positive and Unlabeled Data without Explicit Estimation of Class Prior

2020 ◽  
Vol 34 (04) ◽  
pp. 6762-6769
Author(s):  
Chenguang Zhang ◽  
Yuexian Hou ◽  
Yan Zhang
Keyword(s):  
Unlabeled Data ◽  
Conditional Probability Density ◽  
Classification Problems ◽  
Restricted Boltzmann Machines ◽  
Bhattacharyya Coefficient ◽  
New Strategy ◽  
Positive Class ◽  
Negative Class ◽  
Standard Classification ◽  
Synthetic Datasets

Learning a classifier from positive and unlabeled data may occur in various applications. It differs from the standard classification problems by the absence of labeled negative examples in the training set. So far, two main strategies have typically been used for this issue: the likely negative examplesbased strategy and the class prior-based strategy, in which the likely negative examples or the class prior is required to be obtained in a preprocessing step. In this paper, a new strategy based on the Bhattacharyya coefficient is put forward, which formalizes this learning problem as an optimization problem and does not need a preprocessing step. We first show that with the given positive class conditional probability density function (PDF) and the mixture PDF of both the positive class and the negative class, the class prior can be estimated by minimizing the Bhattacharyya coefficient of the positive class with respect to the negative class. We then show how to use this result in an implicit mixture model of restricted Boltzmann machines to estimate the positive class conditional PDF and the negative class conditional PDF directly to obtain a classifier without the explicit estimation of the class prior. Many experiments on real and synthetic datasets illustrated the superiority of the proposed approach.

Semi-Supervised Learning

2011 ◽  
pp. 1022-1027
Author(s):  
Tobias Scheffer
Keyword(s):  
Supervised Learning ◽  
Supervised Classification ◽  
Unlabeled Data ◽  
Training Data ◽  
Classification Algorithms ◽  
Classification Problems

For many classification problems, unlabeled training data are inexpensive and readily available, whereas labeling training data imposes costs. Semi-supervised classification algorithms aim at utilizing information contained in unlabeled data in addition to the (few) labeled data.

Sign in / Sign up

Export Citation Format

Share Document