scholarly journals High-dimensional semi-supervised learning: in search of optimal inference of the mean

Biometrika ◽  
2021 ◽  
Author(s):  
Yuqian Zhang ◽  
Jelena Bradic
Keyword(s):  
Supervised Learning ◽  
Semiparametric Models ◽  
High Dimensional ◽  
Robust Estimator ◽  
Consistent Estimation ◽  
Heterogeneous Treatment Effects ◽  
Double Robust ◽  
Fundamental Challenge ◽  
The Mean ◽  
Linear And Nonlinear

Abstract A fundamental challenge in semi-supervised learning lies in the observed data’s disproportional size when compared with the size of the data collected with missing outcomes. An implicit understanding is that the dataset with missing outcomes, being significantly larger, ought to improve estimation and inference. However, it is unclear to what extent this is correct. We illustrate one clear benefit: root-n inference of the outcome’s mean is possible while only requiring a consistent estimation of the outcome, possibly at a rate slower than root-n. This is achieved by a novel k-fold cross-fitted, double robust estimator. We discuss both linear and nonlinear outcomes. Such an estimator is particularly suited for models that naturally do not admit root-n consistency, such as high-dimensional, nonparametric, or semiparametric models. We apply our methods to the heterogeneous treatment effects.

scholarly journals Mean Empirical Likelihood Inference for Response Mean with Data Missing at Random

2020 ◽  
Vol 2020 ◽  
pp. 1-12
Author(s):  
Hanji He ◽  
Guangming Deng
Keyword(s):  
Empirical Likelihood ◽  
Missing At Random ◽  
Confidence Regions ◽  
Likelihood Inference ◽  
High Dimensional ◽  
Finite Sample ◽  
The Mean ◽  
Data Missing ◽  
Consistency And Asymptotic Normality ◽  
The Impact

We extend the mean empirical likelihood inference for response mean with data missing at random. The empirical likelihood ratio confidence regions are poor when the response is missing at random, especially when the covariate is high-dimensional and the sample size is small. Hence, we develop three bias-corrected mean empirical likelihood approaches to obtain efficient inference for response mean. As to three bias-corrected estimating equations, we get a new set by producing a pairwise-mean dataset. The method can increase the size of the sample for estimation and reduce the impact of the dimensional curse. Consistency and asymptotic normality of the maximum mean empirical likelihood estimators are established. The finite sample performance of the proposed estimators is presented through simulation, and an application to the Boston Housing dataset is shown.

scholarly journals Multi-Scale Aggregation Graph Neural Networks Based on Feature Similarity for Semi-Supervised Learning

Entropy ◽  
2021 ◽  
Vol 23 (4) ◽  
pp. 403
Author(s):  
Xun Zhang ◽  
Lanyan Yang ◽  
Bin Zhang ◽  
Ying Liu ◽  
Dong Jiang ◽  
...  
Keyword(s):  
Neural Network ◽  
Supervised Learning ◽  
Structured Data ◽  
Similarity Matrix ◽  
Scale Feature ◽  
Multi Scale ◽  
Adjacent Node ◽  
Feature Similarity ◽  
The Mean ◽  
Graph Neural Networks

The problem of extracting meaningful data through graph analysis spans a range of different fields, such as social networks, knowledge graphs, citation networks, the World Wide Web, and so on. As increasingly structured data become available, the importance of being able to effectively mine and learn from such data continues to grow. In this paper, we propose the multi-scale aggregation graph neural network based on feature similarity (MAGN), a novel graph neural network defined in the vertex domain. Our model provides a simple and general semi-supervised learning method for graph-structured data, in which only a very small part of the data is labeled as the training set. We first construct a similarity matrix by calculating the similarity of original features between all adjacent node pairs, and then generate a set of feature extractors utilizing the similarity matrix to perform multi-scale feature propagation on graphs. The output of multi-scale feature propagation is finally aggregated by using the mean-pooling operation. Our method aims to improve the model representation ability via multi-scale neighborhood aggregation based on feature similarity. Extensive experimental evaluation on various open benchmarks shows the competitive performance of our method compared to a variety of popular architectures.

scholarly journals Tropical Balls and Its Applications to K Nearest Neighbor over the Space of Phylogenetic Trees

Mathematics ◽  
2021 ◽  
Vol 9 (7) ◽  
pp. 779
Author(s):  
Ruriko Yoshida
Keyword(s):  
Supervised Learning ◽  
Phylogenetic Trees ◽  
Nearest Neighbor ◽  
Nearest Neighbors ◽  
High Dimensional ◽  
Learning Method ◽  
Dimensional Vector ◽  
K Nearest Neighbor ◽  
K Nearest Neighbors

A tropical ball is a ball defined by the tropical metric over the tropical projective torus. In this paper we show several properties of tropical balls over the tropical projective torus and also over the space of phylogenetic trees with a given set of leaf labels. Then we discuss its application to the K nearest neighbors (KNN) algorithm, a supervised learning method used to classify a high-dimensional vector into given categories by looking at a ball centered at the vector, which contains K vectors in the space.

Robustness to Parametric Assumptions in Missing Data Models

2011 ◽  
Vol 101 (3) ◽  
pp. 538-543 ◽  
Author(s):  
Bryan S Graham ◽  
Keisuke Hirano
Keyword(s):  
Empirical Bayes ◽  
Missing At Random ◽  
Data Driven ◽  
Robust Estimator ◽  
Sampling Variance ◽  
Bayes Estimators ◽  
Cell Means ◽  
Double Robust ◽  
Empirical Bayes Estimators ◽  
Empirical Bayes Estimator

We consider estimation of population averages when data are missing at random. If some cells contain few observations, there can be substantial gains from imposing parametric restrictions on the cell means, but there is also a danger of misspecification. We develop a simple empirical Bayes estimator, which combines parametric and unadjusted estimates of cell means in a data-driven way. We also consider ways to use knowledge of the form of the propensity score to increase robustness. We develop an empirical Bayes extension of a double robust estimator. In a small simulation study, the empirical Bayes estimators perform well. They are similar to fully nonparametric methods and robust to misspecification when cells are moderate to large in size, and when cells are small they maintain the benefits of parametric methods and can have lower sampling variance.

scholarly journals Assembly Quality Detection Based on Class-Imbalanced Semi-Supervised Learning

2021 ◽  
Vol 11 (21) ◽  
pp. 10373
Author(s):  
Zichen Lu ◽  
Jiabin Jiang ◽  
Pin Cao ◽  
Yongying Yang
Keyword(s):  
Deep Learning ◽  
Supervised Learning ◽  
Class Imbalance ◽  
Loss Functions ◽  
Quality Inspection ◽  
Data Annotation ◽  
Quality Detection ◽  
High Classification Accuracy ◽  
Total Data ◽  
The Mean

Due to the imperfect assembly process, the unqualified assembly of a missing gasket or lead seal will affect the product’s performance and possibly cause safety accidents. Machine vision method based on deep learning has been widely used in quality inspection. Semi-supervised learning (SSL) has been applied in training deep learning models to reduce the burden of data annotation. The dataset obtained from the production line tends to be class-imbalanced because the assemblies are qualified in most cases. However, most SSL methods suffer from lower performance in class-imbalanced datasets. Therefore, we propose a new semi-supervised algorithm that achieves high classification accuracy on the class-imbalanced assembly dataset with limited labeled data. Based on the mean teacher algorithm, the proposed algorithm uses certainty to select reliable teacher predictions for student learning dynamically, and loss functions are modified to improve the model’s robustness against class imbalance. Results show that when only 10% of the total data are labeled, and the imbalance rate is 5.3, the proposed method can improve the accuracy from 85.34% to 93.67% compared to supervised learning. When the amount of annotated data accounts for 20%, the accuracy can reach 98.83%.

scholarly journals Semi-Supervised Deep Learning for High-Dimensional Uncertainty Quantification

2020 ◽  
Author(s):  
Zequn Wang ◽  
Mingyang Li
Keyword(s):  
Uncertainty Quantification ◽  
Reliability Analysis ◽  
Supervised Learning ◽  
Dimensional Space ◽  
Limit State ◽  
Failure Surface ◽  
Simulation Method ◽  
High Dimensional ◽  
State Function ◽  
Latent Space

Abstract Conventional uncertainty quantification methods usually lacks the capability of dealing with high-dimensional problems due to the curse of dimensionality. This paper presents a semi-supervised learning framework for dimension reduction and reliability analysis. An autoencoder is first adopted for mapping the high-dimensional space into a low-dimensional latent space, which contains a distinguishable failure surface. Then a deep feedforward neural network (DFN) is utilized to learn the mapping relationship and reconstruct the latent space, while the Gaussian process (GP) modeling technique is used to build the surrogate model of the transformed limit state function. During the training process of the DFN, the discrepancy between the actual and reconstructed latent space is minimized through semi-supervised learning for ensuring the accuracy. Both labeled and unlabeled samples are utilized for defining the loss function of the DFN. Evolutionary algorithm is adopted to train the DFN, then the Monte Carlo simulation method is used for uncertainty quantification and reliability analysis based on the proposed framework. The effectiveness is demonstrated through a mathematical example.

scholarly journals Survey on Feature Transformation Techniques for Data Streams

2020 ◽  
Author(s):  
Maroua Bahri ◽  
Albert Bifet ◽  
Silviu Maniu ◽  
Heitor Murilo Gomes
Keyword(s):  
Machine Learning ◽  
Data Streams ◽  
Large Data ◽  
High Dimensional ◽  
Feature Transformation ◽  
Transformation Techniques ◽  
Computational Costs ◽  
The Past ◽  
Fundamental Challenge ◽  
Mining Algorithms

Mining high-dimensional data streams poses a fundamental challenge to machine learning as the presence of high numbers of attributes can remarkably degrade any mining task's performance. In the past several years, dimension reduction (DR) approaches have been successfully applied for different purposes (e.g., visualization). Due to their high-computational costs and numerous passes over large data, these approaches pose a hindrance when processing infinite data streams that are potentially high-dimensional. The latter increases the resource-usage of algorithms that could suffer from the curse of dimensionality. To cope with these issues, some techniques for incremental DR have been proposed. In this paper, we provide a survey on reduction approaches designed to handle data streams and highlight the key benefits of using these approaches for stream mining algorithms.

scholarly journals Facetto: Combining Unsupervised and Supervised Learning for Hierarchical Phenotype Analysis in Multi-Channel Image Data

2019 ◽  
Author(s):  
Robert Krueger ◽  
Johanna Beyer ◽  
Won-Dong Jang ◽  
Nam Wook Kim ◽  
Artem Sokolov ◽  
...  
Keyword(s):  
Supervised Learning ◽  
Visual Analytics ◽  
Cancer Biology ◽  
Large Scale ◽  
Hierarchical Structures ◽  
Image Data ◽  
Automated Analysis ◽  
Cell Types ◽  
High Dimensional ◽  
Exploration Process

AbstractFacetto is a scalable visual analytics application that is used to discover single-cell phenotypes in high-dimensional multi-channel microscopy images of human tumors and tissues. Such images represent the cutting edge of digital histology and promise to revolutionize how diseases such as cancer are studied, diagnosed, and treated. Highly multiplexed tissue images are complex, comprising 109or more pixels, 60-plus channels, and millions of individual cells. This makes manual analysis challenging and error-prone. Existing automated approaches are also inadequate, in large part, because they are unable to effectively exploit the deep knowledge of human tissue biology available to anatomic pathologists. To overcome these challenges, Facetto enables a semi-automated analysis of cell types and states. It integrates unsupervised and supervised learning into the image and feature exploration process and offers tools for analytical provenance. Experts can cluster the data to discover new types of cancer and immune cells and use clustering results to train a convolutional neural network that classifies new cells accordingly. Likewise, the output of classifiers can be clustered to discover aggregate patterns and phenotype subsets. We also introduce a new hierarchical approach to keep track of analysis steps and data subsets created by users; this assists in the identification of cell types. Users can build phenotype trees and interact with the resulting hierarchical structures of both high-dimensional feature and image spaces. We report on use-cases in which domain scientists explore various large-scale fluorescence imaging datasets. We demonstrate how Facetto assists users in steering the clustering and classification process, inspecting analysis results, and gaining new scientific insights into cancer biology.

Semi-supervised learning using multiple one-dimensional embedding based adaptive interpolation

2016 ◽  
Vol 14 (02) ◽  
pp. 1640002 ◽  
Author(s):  
Jianzhong Wang
Keyword(s):  
Supervised Learning ◽  
High Dimensional ◽  
One Dimensional ◽  
Large Size ◽  
Adaptive Interpolation

We propose a novel semi-supervised learning (SSL) scheme using adaptive interpolation on multiple one-dimensional (1D) embedded data. For a given high-dimensional dataset, we smoothly map it onto several different 1D sequences, so that the labeled subset is converted to a 1D subset for each of these sequences. Applying the cubic interpolation of the labeled subset, we obtain a subset of unlabeled points, which are assigned to the same label in all interpolations. Selecting a proportion of these points at random and adding them to the current labeled subset, we build a larger labeled subset for the next interpolation. Repeating the embedding and interpolation, we enlarge the labeled subset gradually, and finally reach a labeled set with a reasonable large size, based on which the final classifier is constructed. We explore the use of the proposed scheme in the classification of handwritten digits and show promising results.

Sign in / Sign up

Export Citation Format

Share Document