Risk of Selection of Irrelevant Features from High-Dimensional Data with Small Sample Size

2015 ◽  
pp. 399-405
Author(s):  
Henryk Maciejewski
Keyword(s):  
Sample Size ◽  
Small Sample Size ◽  
High Dimensional Data ◽  
Small Sample ◽  
High Dimensional ◽  
Selection Of

scholarly journals -Plot for Testing Spherical Symmetry for High-Dimensional Data with a Small Sample Size

2012 ◽  
Vol 2012 ◽  
pp. 1-18
Author(s):  
Jiajuan Liang
Keyword(s):  
Sample Size ◽  
Spherical Symmetry ◽  
Graphical Method ◽  
Small Sample Size ◽  
High Dimensional Data ◽  
Monte Carlo Study ◽  
Real Data ◽  
Small Sample ◽  
High Dimensional ◽  
Data Set

High-dimensional data with a small sample size, such as microarray data and image data, are commonly encountered in some practical problems for which many variables have to be measured but it is too costly or time consuming to repeat the measurements for many times. Analysis of this kind of data poses a great challenge for statisticians. In this paper, we develop a new graphical method for testing spherical symmetry that is especially suitable for high-dimensional data with small sample size. The new graphical method associated with the local acceptance regions can provide a quick visual perception on the assumption of spherical symmetry. The performance of the new graphical method is demonstrated by a Monte Carlo study and illustrated by a real data set.

scholarly journals An Ensemble Classification Method for High-Dimensional Data Using Neighborhood Rough Set

Complexity ◽  
2021 ◽  
Vol 2021 ◽  
pp. 1-12
Author(s):  
Jing Zhang ◽  
Guang Lu ◽  
Jiaquan Li ◽  
Chuanwen Li
Keyword(s):  
Feature Selection ◽  
Rough Set ◽  
Small Sample Size ◽  
High Dimensional Data ◽  
Classification Performance ◽  
Small Sample ◽  
Ensemble Classification ◽  
High Dimensional ◽  
Sample Classification ◽  
Neighborhood Rough Set

Mining useful knowledge from high-dimensional data is a hot research topic. Efficient and effective sample classification and feature selection are challenging tasks due to high dimensionality and small sample size of microarray data. Feature selection is necessary in the process of constructing the model to reduce time and space consumption. Therefore, a feature selection model based on prior knowledge and rough set is proposed. Pathway knowledge is used to select feature subsets, and rough set based on intersection neighborhood is then used to select important feature in each subset, since it can select features without redundancy and deals with numerical features directly. In order to improve the diversity among base classifiers and the efficiency of classification, it is necessary to select part of base classifiers. Classifiers are grouped into several clusters by k-means clustering using the proposed combination distance of Kappa-based diversity and accuracy. The base classifier with the best classification performance in each cluster will be selected to generate the final ensemble model. Experimental results on three Arabidopsis thaliana stress response datasets showed that the proposed method achieved better classification performance than existing ensemble models.

scholarly journals Filtering high-dimensional methylation marks with extremely small sample size: an application to gastric cancer data

2021 ◽  
Author(s):  
Xin Chen ◽  
Qingrun Zhang ◽  
Thierry Chekouo
Keyword(s):  
Gastric Cancer ◽  
Dna Methylation ◽  
Sample Size ◽  
Small Sample Size ◽  
Small Sample ◽  
Differential Methylation ◽  
High Dimensional ◽  
Cancer Data ◽  
Cancer Pathogenesis ◽  
A Genome

Abstract Background: DNA methylations in critical regions are highly involved in cancer pathogenesis and drug response. However, to identify causal methylations out of a large number of potential polymorphic DNA methylation sites is challenging. This high-dimensional data brings two obstacles: first, many established statistical models are not scalable to so many features; second, multiple-test and overfitting become serious. To this end, a method to quickly filter candidate sites to narrow down targets for downstream analyses is urgently needed. Methods: BACkPAy is a pre-screening Bayesian approach to detect biological meaningful clusters of potential differential methylation levels with small sample size. BACkPAy prioritizes potentially important biomarkers by the Bayesian false discovery rate (FDR) approach. It filters non-informative sites (i.e. non-differential) with flat methylation pattern levels accross experimental conditions. In this work, we applied BACkPAy to a genome-wide methylation dataset with 3 tissue types and each type contains 3 gastric cancer samples. We also applied LIMMA (Linear Models for Microarray and RNA-Seq Data) to compare its results with what we achieved by BACkPAy. Then, Cox proportional hazards regression models were utilized to visualize prognostics significant markers with The Cancer Genome Atlas (TCGA) data for survival analysis. Results: Using BACkPAy, we identified 8 biological meaningful clusters/groups of differential probes from the DNA methylation dataset. Using TCGA data, we also identified five prognostic genes (i.e. predictive to the progression of gastric cancer) that contain some differential methylation probes, whereas no significant results was identified using the Benjamin-Hochberg FDR in LIMMA. Conclusions: We showed the importance of using BACkPAy for the analysis of DNA methylation data with extremely small sample size in gastric cancer. We revealed that RDH13, CLDN11, TMTC1, UCHL1 and FOXP2 can serve as predictive biomarkers for gastric cancer treatment and the promoter methylation level of these five genes in serum could have prognostic and diagnostic functions in gastric cancer patients.

scholarly journals Filtering High-Dimensional Methylation Marks With Extremely Small Sample Size: An Application to Gastric Cancer Data

2021 ◽  
Vol 12 ◽  
Author(s):  
Xin Chen ◽  
Qingrun Zhang ◽  
Thierry Chekouo
Keyword(s):  
Gastric Cancer ◽  
Dna Methylation ◽  
Sample Size ◽  
Small Sample Size ◽  
Small Sample ◽  
Differential Methylation ◽  
High Dimensional ◽  
Cancer Data ◽  
Cancer Pathogenesis ◽  
A Genome

DNA methylations in critical regions are highly involved in cancer pathogenesis and drug response. However, to identify causal methylations out of a large number of potential polymorphic DNA methylation sites is challenging. This high-dimensional data brings two obstacles: first, many established statistical models are not scalable to so many features; second, multiple-test and overfitting become serious. To this end, a method to quickly filter candidate sites to narrow down targets for downstream analyses is urgently needed. BACkPAy is a pre-screening Bayesian approach to detect biological meaningful patterns of potential differential methylation levels with small sample size. BACkPAy prioritizes potentially important biomarkers by the Bayesian false discovery rate (FDR) approach. It filters non-informative sites (i.e., non-differential) with flat methylation pattern levels across experimental conditions. In this work, we applied BACkPAy to a genome-wide methylation dataset with three tissue types and each type contains three gastric cancer samples. We also applied LIMMA (Linear Models for Microarray and RNA-Seq Data) to compare its results with what we achieved by BACkPAy. Then, Cox proportional hazards regression models were utilized to visualize prognostics significant markers with The Cancer Genome Atlas (TCGA) data for survival analysis. Using BACkPAy, we identified eight biological meaningful patterns/groups of differential probes from the DNA methylation dataset. Using TCGA data, we also identified five prognostic genes (i.e., predictive to the progression of gastric cancer) that contain some differential methylation probes, whereas no significant results was identified using the Benjamin-Hochberg FDR in LIMMA. We showed the importance of using BACkPAy for the analysis of DNA methylation data with extremely small sample size in gastric cancer. We revealed that RDH13, CLDN11, TMTC1, UCHL1, and FOXP2 can serve as predictive biomarkers for gastric cancer treatment and the promoter methylation level of these five genes in serum could have prognostic and diagnostic functions in gastric cancer patients.

scholarly journals Shrinkage-based diagonal Hotelling’s tests for high-dimensional small sample size data

2016 ◽  
Vol 143 ◽  
pp. 127-142 ◽  
Author(s):  
Kai Dong ◽  
Herbert Pang ◽  
Tiejun Tong ◽  
Marc G. Genton
Keyword(s):  
Sample Size ◽  
Small Sample Size ◽  
Small Sample ◽  
High Dimensional ◽  
Size Data

scholarly journals Bandwidth Detection of Graph Signals with a Small Sample Size

Sensors ◽  
2020 ◽  
Vol 21 (1) ◽  
pp. 146
Author(s):  
Xuan Xie ◽  
Hui Feng ◽  
Bo Hu
Keyword(s):  
Numerical Analysis ◽  
Sample Size ◽  
Null Hypothesis ◽  
Small Sample Size ◽  
Score Test ◽  
Small Sample ◽  
High Dimensional ◽  
Multi Stage ◽  
Spectral Components ◽  
The Given

Bandwidth is the crucial knowledge to sampling, reconstruction or estimation of the graph signal (GS). However, it is typically unknown in practice. In this paper, we focus on detecting the bandwidth of bandlimited GS with a small sample size, where the number of spectral components of GS to be tested may greatly exceed the sample size. To control the significance of the result, the detection procedure is implemented by multi-stage testing. In each stage, a Bayesian score test, which introduces a prior to the spectral components, is adopted to face the high dimensional challenge. By setting different priors in each stage, we make the test more powerful against alternatives that have similar bandwidth to the null hypothesis. We prove that the Bayesian score test is locally most powerful in expectation against the alternatives following the given prior. Finally, numerical analysis shows that our method has a good performance in bandwidth detection and is robust to the noise.

Sign in / Sign up

Export Citation Format

Share Document