Using visual statistical inference to better understand random class separations in high dimension, low sample size data

2014 ◽  
Vol 30 (2) ◽  
pp. 293-316 ◽  
Author(s):  
Niladri Roy Chowdhury ◽  
Dianne Cook ◽  
Heike Hofmann ◽  
Mahbubul Majumder ◽  
Eun-Kyung Lee ◽  
...  
Keyword(s):  
Sample Size ◽  
Statistical Inference ◽  
High Dimension ◽  
Size Data

Statistical inference for high-dimension, low-sample-size data

2017 ◽  
Vol 30 (2) ◽  
pp. 137-158
Author(s):  
Makoto Aoshima ◽  
Kazuyoshi Yata
Keyword(s):  
Sample Size ◽  
Statistical Inference ◽  
High Dimension ◽  
Size Data

scholarly journals On some graph-based two-sample tests for high dimension, low sample size data

2019 ◽  
Vol 109 (2) ◽  
pp. 279-306
Author(s):  
Soham Sarkar ◽  
Rahul Biswas ◽  
Anil K. Ghosh
Keyword(s):  
Sample Size ◽  
High Dimension ◽  
Size Data

scholarly journals Deep Neural Networks for High Dimension, Low Sample Size Data

2017 ◽  
Author(s):  
Bo Liu ◽  
Ying Wei ◽  
Yu Zhang ◽  
Qiang Yang
Keyword(s):  
Neural Networks ◽  
Sample Size ◽  
High Dimension ◽  
Deep Neural Networks ◽  
Genetic Data ◽  
High Dimensional ◽  
Large Sample Size ◽  
Prediction Problem ◽  
The Stability ◽  
Size Data

Deep neural networks (DNN) have achieved breakthroughs in applications with large sample size. However, when facing high dimension, low sample size (HDLSS) data, such as the phenotype prediction problem using genetic data in bioinformatics, DNN suffers from overfitting and high-variance gradients. In this paper, we propose a DNN model tailored for the HDLSS data, named Deep Neural Pursuit (DNP). DNP selects a subset of high dimensional features for the alleviation of overfitting and takes the average over multiple dropouts to calculate gradients with low variance. As the first DNN method applied on the HDLSS data, DNP enjoys the advantages of the high nonlinearity, the robustness to high dimensionality, the capability of learning from a small number of samples, the stability in feature selection, and the end-to-end training. We demonstrate these advantages of DNP via empirical results on both synthetic and real-world biological datasets.

Sign in / Sign up

Export Citation Format

Share Document