Semi-Supervised Learning Exploiting Robust Loss Function for Sparse Labeled Data

Self-supervised learning and pre-training strategies have developed over the last few years especially for Convolutional Neural Networks (CNNs). Recently application of such methods can also be noticed for Graph Neural Networks (GNNs). In this paper, we have used a graph based self-supervised learning strategy with different loss functions (Barlow Twins[? ], HSIC[? ], VICReg[? ]) which have shown promising results when applied with CNNs previously. We have also proposed a hybrid loss function combining the advantages of VICReg and HSIC and called it as VICRegHSIC. The performance of these aforementioned methods have been compared when applied to two different datasets namely MUTAG and PROTEINS. Moreover, the impact of different batch sizes, projector dimensions and data augmentation strategies have also been explored. The results are preliminary and we will be continuing to explore with other datasets.

Download Full-text

PAC-Bayes Unleashed: Generalisation Bounds with Unbounded Losses

Entropy ◽

10.3390/e23101330 ◽

2021 ◽

Vol 23 (10) ◽

pp. 1330

Author(s):

Maxime Haddouche ◽

Benjamin Guedj ◽

Omar Rivasplata ◽

John Shawe-Taylor

Keyword(s):

Linear Regression ◽

Supervised Learning ◽

Loss Function ◽

Loss Functions ◽

Learning Problems ◽

Regression Problem ◽

Learning Framework ◽

Actual Computation ◽

Linear Regression Problem

We present new PAC-Bayesian generalisation bounds for learning problems with unbounded loss functions. This extends the relevance and applicability of the PAC-Bayes learning framework, where most of the existing literature focuses on supervised learning problems with a bounded loss function (typically assumed to take values in the interval [0;1]). In order to relax this classical assumption, we propose to allow the range of the loss to depend on each predictor. This relaxation is captured by our new notion of HYPothesis-dependent rangE (HYPE). Based on this, we derive a novel PAC-Bayesian generalisation bound for unbounded loss functions, and we instantiate it on a linear regression problem. To make our theory usable by the largest audience possible, we include discussions on actual computation, practicality and limitations of our assumptions.

Download Full-text

Online Active Learning of Reject Option Classifiers

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v34i04.6019 ◽

2020 ◽

Vol 34 (04) ◽

pp. 5652-5659

Author(s):

Kulin Shah ◽

Naresh Manwani

Keyword(s):

Machine Learning ◽

Active Learning ◽

Supervised Learning ◽

Loss Function ◽

Learning Algorithm ◽

Binary Classification ◽

Experimental Results ◽

Reject Option ◽

Novel Algorithms ◽

Ramp Loss Function

Active learning is an important technique to reduce the number of labeled examples in supervised learning. Active learning for binary classification has been well addressed in machine learning. However, active learning of the reject option classifier remains unaddressed. In this paper, we propose novel algorithms for active learning of reject option classifiers. We develop an active learning algorithm using double ramp loss function. We provide mistake bounds for this algorithm. We also propose a new loss function called double sigmoid loss function for reject option and corresponding active learning algorithm. We offer a convergence guarantee for this algorithm. We provide extensive experimental results to show the effectiveness of the proposed algorithms. The proposed algorithms efficiently reduce the number of label examples required.

Download Full-text

Semantic and Generalized Entropy Loss Functions for Semi-Supervised Deep Learning

Entropy ◽

10.3390/e22030334 ◽

2020 ◽

Vol 22 (3) ◽

pp. 334

Author(s):

Krzysztof Gajowniczek ◽

Yitao Liang ◽

Tal Friedman ◽

Tomasz Ząbkowski ◽

Guy Van den Broeck

Keyword(s):

Neural Network ◽

Supervised Learning ◽

Loss Function ◽

Practical Importance ◽

Loss Functions ◽

Generalized Entropy ◽

Entropy Loss ◽

Additional Information ◽

Label Information ◽

Benchmark Datasets

The increasing size of modern datasets combined with the difficulty of obtaining real label information (e.g., class) has made semi-supervised learning a problem of considerable practical importance in modern data analysis. Semi-supervised learning is supervised learning with additional information on the distribution of the examples or, simultaneously, an extension of unsupervised learning guided by some constraints. In this article we present a methodology that bridges between artificial neural network output vectors and logical constraints. In order to do this, we present a semantic loss function and a generalized entropy loss function (Rényi entropy) that capture how close the neural network is to satisfying the constraints on its output. Our methods are intended to be generally applicable and compatible with any feedforward neural network. Therefore, the semantic loss and generalized entropy loss are simply a regularization term that can be directly plugged into an existing loss function. We evaluate our methodology over an artificially simulated dataset and two commonly used benchmark datasets which are MNIST and Fashion-MNIST to assess the relation between the analyzed loss functions and the influence of the various input and tuning parameters on the classification accuracy. The experimental evaluation shows that both losses effectively guide the learner to achieve (near-) state-of-the-art results on semi-supervised multiclass classification.

Download Full-text

Supervised learning as an inverse problem based on non-smooth loss function

Knowledge and Information Systems ◽

10.1007/s10115-020-01439-2 ◽

2020 ◽

Vol 62 (8) ◽

pp. 3039-3058

Author(s):

Soufiane Lyaqini ◽

Mohamed Quafafou ◽

Mourad Nachaoui ◽

Abdelkrim Chakib

Keyword(s):

Inverse Problem ◽

Supervised Learning ◽

Loss Function

Download Full-text

Valence electron spectroscopy of inhomogeneous media

Proceedings, annual meeting, Electron Microscopy Society of America ◽

10.1017/s0424820100130389 ◽

1992 ◽

Vol 50 (2) ◽

pp. 1150-1151

Author(s):

A. Howie ◽

D.W. McComb

Keyword(s):

Specific Gravity ◽

Loss Function ◽

Electron Spectroscopy ◽

Valence Electron ◽

Peak Height ◽

Loss Functions ◽

Loss Peak ◽

High Energies ◽

Valence Electron Density ◽

Electron Microscopist

The bulk loss function Im(-l/ε (ω)), a well established tool for the interpretation of valence loss spectra, is being progressively adapted to the wide variety of inhomogeneous samples of interest to the electron microscopist. Proportionality between n, the local valence electron density, and ε-1 (Sellmeyer's equation) has sometimes been assumed but may not be valid even in homogeneous samples. Figs. 1 and 2 show the experimentally measured bulk loss functions for three pure silicates of different specific gravity ρ - quartz (ρ = 2.66), coesite (ρ = 2.93) and a zeolite (ρ = 1.79). Clearly, despite the substantial differences in density, the shift of the prominent loss peak is very small and far less than that predicted by scaling e for quartz with Sellmeyer's equation or even the somewhat smaller shift given by the Clausius-Mossotti (CM) relation which assumes proportionality between n (or ρ in this case) and (ε - 1)/(ε + 2). Both theories overestimate the rise in the peak height for coesite and underestimate the increase at high energies.

Download Full-text

Forecasting in the Presence of an Asymmetric Loss Function

PsycEXTRA Dataset ◽

10.1037/e616342011-036 ◽

1991 ◽

Author(s):

Michael Lawrence ◽

Peter Ayton

Keyword(s):

Loss Function ◽

Asymmetric Loss ◽

Asymmetric Loss Function

Download Full-text

Automatic accounting of Baikal diatomic algae: approaches and prospects

Issues of modern algology (Вопросы современной альгологии) ◽

10.33624/2311-0147-2019-2(20)-295-299 ◽

2019 ◽

pp. 295-299

Author(s):

Кonstantin А. Elshin ◽

Еlena I. Molchanova ◽

Мarina V. Usoltseva ◽

Yelena V. Likhoshway

Keyword(s):

Object Detection ◽

Loss Function ◽

Classification Accuracy ◽

Diatom Species ◽

Bounding Box ◽

Synedra Acus ◽

And Training

Using the TensorFlow Object Detection API, an approach to identifying and registering Baikal diatom species Synedra acus subsp. radians has been tested. As a result, a set of images was formed and training was conducted. It is shown that аfter 15000 training iterations, the total value of the loss function was obtained equal to 0,04. At the same time, the classification accuracy is equal to 95%, and the accuracy of construction of the bounding box is also equal to 95%.

Download Full-text