Duality Between Learning Machines: A Bridge Between Supervised and Unsupervised Learning

1994 ◽  
Vol 6 (3) ◽  
pp. 491-508 ◽  
Author(s):  
J.-P. Nadal ◽  
N. Parga
Keyword(s):  
Theoretical Analysis ◽  
Unsupervised Learning ◽  
Supervised Learning ◽  
Learning Task ◽  
Maximum Information ◽  
Learning Machines ◽  
Learning Tasks ◽  
Supervised And Unsupervised Learning ◽  
Second Dual

We exhibit a duality between two perceptrons that allows us to compare the theoretical analysis of supervised and unsupervised learning tasks. The first perceptron has one output and is asked to learn a classification of p patterns. The second (dual) perceptron has p outputs and is asked to transmit as much information as possible on a distribution of inputs. We show in particular that the maximum information that can be stored in the couplings for the supervised learning task is equal to the maximum information that can be transmitted by the dual perceptron.

LAMDA-HAD, an Extension to the LAMDA Classifier in the Context of Supervised Learning

2020 ◽  
Vol 19 (01) ◽  
pp. 283-316 ◽  
Author(s):  
Luis Morales ◽  
José Aguilar ◽  
Danilo Chávez ◽  
Claudia Isaza
Keyword(s):  
Data Analysis ◽  
Statistical Analysis ◽  
Unsupervised Learning ◽  
Supervised Learning ◽  
Learning Algorithm ◽  
New Approach ◽  
Supervised And Unsupervised Learning ◽  
Training Stage

This paper proposes a new approach to improve the performance of Learning Algorithm for Multivariable Data Analysis (LAMDA). This algorithm can be used for supervised and unsupervised learning, based on the calculation of the Global Adequacy Degree (GAD) of one individual to a class, through the contributions of all its descriptors. LAMDA has the capability of creating new classes after the training stage. If an individual does not have enough similarity to the preexisting classes, it is evaluated with respect to a threshold called the Non-Informative Class (NIC), this being the novelty of the algorithm. However, LAMDA has problems making good classifications, either because the NIC is constant for all classes, or because the GAD calculation is unreliable. In this work, its efficiency is improved by two strategies, the first one, by the calculation of adaptable NICs for each class, which prevents that correctly classified individuals create new classes; and the second one, by computing the Higher Adequacy Degree (HAD), which grants more robustness to the algorithm. LAMDA-HAD is validated by applying it in different benchmarks and comparing it with LAMDA and other classifiers, through a statistical analysis to determinate the cases in which our algorithm presents a better performance.

scholarly journals Using clustering to aid text classification of single-labelled datasets

2009 ◽  
Author(s):  
Αντωνία Κυριακοπούλου
Keyword(s):  
Unsupervised Learning ◽  
Text Classification ◽  
Data Representation ◽  
Classification Performance ◽  
Critical Research ◽  
Social Bookmarking ◽  
Supervised And Unsupervised Learning ◽  
Text Classifiers ◽  
Concise Representation

Supervised and unsupervised learning have been the focus of critical research in the areas of machine learning and artificial intelligence. In the literature, these two streams flow independently of each other, despite their close conceptual and practical connections. This dissertation demonstrates that unsupervised learning algorithms, i.e. clustering, can provide us with valuable information about the data and help in the creation of high-accuracy text classifiers. In the case of clustering,the aim is to extract a kind of \structure" from a given sample of objects. The reasoning behind this is that if some structure exists in the objects, it is possible to take advantage of this information and find a short description of the data,exploiting the dependence or association between index terms and documents.This concise representation of the whole dataset can be properly incorporated in the existing data representation. The use of prior knowledge about the nature oft he dataset helps in building a more efficient classifier for this set. This approach does not capture all the intricacies of text; however on some domains this technique substantially improves text classification accuracy.In this vein, a study of the interaction between supervised and unsupervised learning has been carried out. We have studied and implemented models that apply clustering in multiple ways and in conjunction with classification to construct robust text classifiers. The extensive experimentation has shown the effectiveness of using clustering to boost text classification performance. Additionally, preliminary experiments on some of the most important applications of text classification such as Spam Mail Filtering, Spam Detection in Social Bookmarking Systems,and Sentence Boundary Disambiguation, have shown promising enhancements by exploiting the proposed models.

scholarly journals selfRL: Two-Level Self-Supervised Transformer Representation Learning for Link Prediction of Heterogeneous Biomedical Networks

2020 ◽  
Author(s):  
Xiaoqi Wang ◽  
Yaning Yang ◽  
Xiangke Liao ◽  
Lenli Li ◽  
Fei Li ◽  
...  
Keyword(s):  
Supervised Learning ◽  
Link Prediction ◽  
Level Structure ◽  
Representation Learning ◽  
Learning Task ◽  
Learning Tasks ◽  
Biomedical Problem ◽  
Great Performance ◽  
Meta Path ◽  
Frame Work

AbstractPredicting potential links in heterogeneous biomedical networks (HBNs) can greatly benefit various important biomedical problem. However, the self-supervised representation learning for link prediction in HBNs has been slightly explored in previous researches. Therefore, this study proposes a two-level self-supervised representation learning, namely selfRL, for link prediction in heterogeneous biomedical networks. The meta path detection-based self-supervised learning task is proposed to learn representation vectors that can capture the global-level structure and semantic feature in HBNs. The vertex entity mask-based self-supervised learning mechanism is designed to enhance local association of vertices. Finally, the representations from two tasks are concatenated to generate high-quality representation vectors. The results of link prediction on six datasets show selfRL outperforms 25 state-of-the-art methods. In particular, selfRL reveals great performance with results close to 1 in terms of AUC and AUPR on the NeoDTI-net dataset. In addition, the PubMed publications demonstrate that nine out of ten drugs screened by selfRL can inhibit the cytokine storm in COVID-19 patients. In summary, selfRL provides a general frame-work that develops self-supervised learning tasks with unlabeled data to obtain promising representations for improving link prediction.

scholarly journals GCIceNet: a graph convolutional network for accurate classification of water phases

2020 ◽  
Vol 22 (45) ◽  
pp. 26340-26350
Author(s):  
QHwan Kim ◽  
Joon-Hyuk Ko ◽  
Sunghoon Kim ◽  
Wonho Jhe
Keyword(s):  
Unsupervised Learning ◽  
Order Parameters ◽  
Water Molecules ◽  
Convolutional Network ◽  
Convolutional Networks ◽  
Supervised And Unsupervised Learning

We develop GCIceNet, which automatically generates machine-based order parameters for classifying the phases of water molecules via supervised and unsupervised learning with graph convolutional networks.

Teacher and learner: Supervised and unsupervised learning in communities

2015 ◽  
Vol 38 ◽  
Author(s):  
Michael G. Shafto ◽  
Colleen M. Seifert
Keyword(s):  
Unsupervised Learning ◽  
Teaching Methods ◽  
Learning Task ◽  
Costs And Benefits ◽  
Optimal Methods ◽  
Supervised And Unsupervised Learning

AbstractHow far can teaching methods go to enhance learning? Optimal methods of teaching have been considered in research on supervised and unsupervised learning. Locally optimal methods are usually hybrids of teaching and self-directed approaches. The costs and benefits of specific methods have been shown to depend on the structure of the learning task, the learners, the teachers, and the environment.

scholarly journals Semi-supervised learning: a brief review

2018 ◽  
Vol 7 (1.8) ◽  
pp. 81 ◽  
Author(s):  
Y C A Padmanabha Reddy ◽  
P Viswanath ◽  
B Eswara Reddy
Keyword(s):  
Unsupervised Learning ◽  
Supervised Learning ◽  
Supervised Classification ◽  
Unlabeled Data ◽  
Application Domain ◽  
Supervised Clustering ◽  
Domain Experts ◽  
Supervised And Unsupervised Learning ◽  
Learning Techniques

Most of the application domain suffers from not having sufficient labeled data whereas unlabeled data is available cheaply. To get labeled instances, it is very difficult because experienced domain experts are required to label the unlabeled data patterns. Semi-supervised learning addresses this problem and act as a half way between supervised and unsupervised learning. This paper addresses few techniques of Semi-supervised learning (SSL) such as self-training, co-training, multi-view learning, TSVMs methods. Traditionally SSL is classified in to Semi-supervised Classification and Semi-supervised Clustering which achieves better accuracy than traditional supervised and unsupervised learning techniques. The paper also addresses the issue of scalability and applications of Semi-supervised learning. 

Sign in / Sign up

Export Citation Format

Share Document