scholarly journals The Eight Steps of Data Analysis: A Graphical Framework to Promote Sound Statistical Analysis

2020 ◽  
Vol 15 (4) ◽  
pp. 1054-1075
Author(s):  
Dustin Fife
Keyword(s):  
Data Analysis ◽  
Statistical Analysis ◽  
Outlier Detection ◽  
Multiple Comparisons ◽  
False Positives ◽  
Small Sample ◽  
False Negatives ◽  
Analysis Strategy ◽  
General Statistical ◽  
Small Sample Sizes

Data analysis is a risky endeavor, particularly among people who are unaware of its dangers. According to some researchers, “statistical conclusions validity” threatens all research subjected to the dark arts of statistical magic. Although traditional statistics classes may advise against certain practices (e.g., multiple comparisons, small sample sizes, violating normality), they may fail to cover others (e.g., outlier detection and violating linearity). More common, perhaps, is that researchers may fail to remember them. In this article, rather than rehashing old warnings and diatribes against this practice or that, I instead advocate a general statistical-analysis strategy. This graphic-based eight-step strategy promises to resolve the majority of statistical traps researchers may fall into—without having to remember large lists of problematic statistical practices. These steps will assist in preventing both false positives and false negatives and yield critical insights about the data that would have otherwise been missed. I conclude with an applied example that shows how the eight steps reveal interesting insights that would not be detected with standard statistical practices.

scholarly journals The Eight Steps of Data Analysis: A Graphical Framework to Promote Sound Statistical Analysis

2019 ◽  
Author(s):  
Dustin Fife
Keyword(s):  
Data Analysis ◽  
Statistical Analysis ◽  
Multiple Comparisons ◽  
Small Sample ◽  
Type I ◽  
Type Ii ◽  
Analysis Strategy ◽  
General Statistical ◽  
Small Sample Sizes ◽  
Type Ii Errors

Data analysis is a risky endeavor, particularly among those unaware of its dangers. In the words of Cook and Campbell (1976; see also Cook, Campbell, and Shadish 2002), “Statistical Conclusions Validity” threatens all experiments that subject themselves to the dark arts of statistical magic. Although traditional statistics classes may advise against certain practices (e.g., multiple comparisons, small sample sizes, violating normality), they may fail to cover others (e.g., outlier detection and violating linearity). More common, perhaps, is that researchers may fail to remember them. In this paper, rather than rehashing old warnings and diatribes against this practice or that, I instead advocate a general statistical analysis strategy. This graphically-based eight step strategy promises to resolve the majority of statistical traps researchers may fall in without having to remember large lists of problematic statistical practices. These steps will assist in preventing both Type I and Type II errors and yield critical insights about the data that would have otherwise been missed. I conclude with an applied example that shows how the eight steps highlight data problems that would not be detected with standard statistical practices.

scholarly journals Seeking genetic signature of radiosensitivity - a novel method for data analysis in case of small sample sizes

2014 ◽  
Vol 11 (Suppl 1) ◽  
pp. S2 ◽  
Author(s):  
Joanna Zyla ◽  
Paul Finnon ◽  
Robert Bulman ◽  
Simon Bouffler ◽  
Christophe Badie ◽  
...  
Keyword(s):  
Data Analysis ◽  
Small Sample ◽  
Sample Sizes ◽  
Genetic Signature ◽  
Novel Method ◽  
Small Sample Sizes

PREFERRED SEQUENCES OF GENETIC EVENTS IN CARCINOGENESIS: QUANTITATIVE ASPECTS OF THE PROBLEM

2001 ◽  
Vol 09 (02) ◽  
pp. 105-121 ◽  
Author(s):  
ANIKO SZABO ◽  
ANDREI YAKOVLEV
Keyword(s):  
Data Analysis ◽  
Tumor Development ◽  
Sampling Bias ◽  
Small Sample ◽  
Mutation Rates ◽  
Mechanistic Models ◽  
Chronological Order ◽  
Observation Process ◽  
Small Sample Sizes ◽  
Genetic Events

In this paper we discuss some natural limitations in quantitative inference about the frequency, correlation and ordering of genetic events occurring in the course of tumor development. We consider a simple, yet frequently used experimental design, under which independent tumors are examined once for the presence/absence of specific mutations of interest. The most typical factors that affect the inference on the chronological order of genetic events are: a possible dependence of mutation rates, the sampling bias that arises from the observation process and small sample sizes. Our results clearly indicate that just these three factors alone may dramatically distort the outcome of data analysis, thereby leading to estimates of limited utility as an underpinning for mechanistic models of carcinogenesis.

scholarly journals A simple correction for non-independent tests

2018 ◽  
Author(s):  
Jaime Derringer
Keyword(s):  
Statistical Significance ◽  
Multiple Comparisons ◽  
Psychological Research ◽  
False Positives ◽  
High Rate ◽  
Effective Number ◽  
False Negatives

Psychologists wrestle with how to best handle multiple comparisons, while maintaining a balance between false positives and false negatives. Undercorrection, such as ignoring the presence of multiple comparisons altogether, is known to yield an unacceptably high rate of false positives. Overcorrection, such as treating all tests as independent when they are not, results in overly conservative evaluations of statistical significance. This tutorial demonstrates $M_{eff}$ correction, a method for adjusting statistical significance thresholds for multiple comparisons, without the assumption of independence of tests. This method, in which the effective number of tests ($M_{eff}$) is estimated from the correlations among the variables being tested, was developed and validated in the field of genetics, but is based on statistical concepts (eigenvalues) that are very familiar to psychologists. $M_{eff}$ correction can be applied in psychological research to balance the necessity of correction for multiple comparisons with the concerns that arise from complex, correlated tests.

What if we ignore the random effects when analyzing RNA-seq data in a multifactor experiment

2016 ◽  
Vol 15 (2) ◽  
Author(s):  
Shiqi Cui ◽  
Tieming Ji ◽  
Jilong Li ◽  
Jianlin Cheng ◽  
Jing Qiu
Keyword(s):  
Data Analysis ◽  
Random Effects ◽  
Early Stage ◽  
Real Data ◽  
Small Sample ◽  
Repeated Measure ◽  
Group Comparison ◽  
Rna Seq ◽  
Multifactor Experiment ◽  
Small Sample Sizes

AbstractIdentifying differentially expressed (DE) genes between different conditions is one of the main goals of RNA-seq data analysis. Although a large amount of RNA-seq data were produced for two-group comparison with small sample sizes at early stage, more and more RNA-seq data are being produced in the setting of complex experimental designs such as split-plot designs and repeated measure designs. Data arising from such experiments are traditionally analyzed by mixed-effects models. Therefore an appropriate statistical approach for analyzing RNA-seq data from such designs should be generalized linear mixed models (GLMM) or similar approaches that allow for random effects. However, common practices for analyzing such data in literature either treat random effects as fixed or completely ignore the experimental design and focus on two-group comparison using partial data. In this paper, we examine the effect of ignoring the random effects when analyzing RNA-seq data. We accomplish this goal by comparing the standard GLMM model to the methods that ignore the random effects through simulation studies and real data analysis. Our studies show that, ignoring random effects in a multi-factor experiment can lead to the increase of the false positives among the top selected genes or lower power when the nominal FDR level is controlled.

scholarly journals Replacement of Biased Estimators with Unbiased Ones in the Case of Student’s t-Distribution and Geary’s Kurtosis

2017 ◽  
Vol 45 (1) ◽  
pp. 23-27
Author(s):  
Gergely Tóth ◽  
Pál Szepesváry
Keyword(s):  
Data Analysis ◽  
Small Sample ◽  
Unbiased Estimation ◽  
Distributed Data ◽  
Size Dependent ◽  
Population Standard Deviation ◽  
Quantitative Results ◽  
Biased Estimators ◽  
Small Sample Sizes ◽  
T Distribution

Abstract The use of biased estimators can be found in some historically and up to now important tools in statistical data analysis. In this paper their replacement with unbiased estimators at least in the case of the estimator of the population standard deviation for normal distributions is proposed. By removing the incoherence from the Student’s t-distribution caused by the biased estimator, a corrected t-distribution may be defined. Although the quantitative results in most data analysis applications are identical for both the original and corrected tdistributions, the use of this last t-distribution is suggested because of its theoretical consistency. Moreover, the frequent qualitative discussion of the t-distribution has come under much criticism, because it concerns artefacts of the biased estimator. In the case of Geary’s kurtosis the same correction results (2/π)1/2 unbiased estimation of kurtosis for normally distributed data that is independent of the size of the sample. It is believed that by removing the sample-size-dependent biased feature, the applicability domain can be expanded to include small sample sizes for some normality tests.

scholarly journals Reproducible research into human chemical communication by cues and pheromones: learning from psychology's renaissance

2020 ◽  
Vol 375 (1800) ◽  
pp. 20190262 ◽  
Author(s):  
Tristram D. Wyatt
Keyword(s):  
Chemical Communication ◽  
Chemical Cues ◽  
False Positives ◽  
Small Sample ◽  
Olfactory Cues ◽  
Reproducible Research ◽  
Olfactory Communication ◽  
Biomedical Sciences ◽  
Human Pheromones ◽  
Small Sample Sizes

Despite the lack of evidence that the ‘putative human pheromones' androstadienone and estratetraenol ever were pheromones, almost 60 studies have claimed ‘significant' results. These are quite possibly false positives and can be best seen as potential examples of the ‘reproducibility crisis', sadly common in the rest of the life and biomedical sciences, which has many instances of whole fields based on false positives. Experiments on the effects of olfactory cues on human behaviour are also at risk of false positives because they look for subtle effects but use small sample sizes. Research on human chemical communication, much of it falling within psychology, would benefit from vigorously adopting the proposals made by psychologists to enable better, more reliable science, with an emphasis on enhancing reproducibility. A key change is the adoption of study pre-registration and/or Registered Reports which will also reduce publication bias. As we are mammals, and chemical communication is important to other mammals, it is likely that chemical cues are important in our behaviour and that humans may have pheromones, but new approaches will be needed to reliably demonstrate them. This article is part of the Theo Murphy meeting issue ‘Olfactory communication in humans’.

Statistical analysis of false positives and false negatives from real traffic with intrusion detection/prevention systems

2012 ◽  
Vol 50 (3) ◽  
pp. 146-154 ◽  
Author(s):  
Cheng-Yuan Ho ◽  
Yuan-Cheng Lai ◽  
I-Wei Chen ◽  
Fu-Yu Wang ◽  
Wei-Hsuan Tai
Keyword(s):  
Statistical Analysis ◽  
Intrusion Detection ◽  
False Positives ◽  
False Negatives ◽  
Real Traffic

scholarly journals Exploring Whether Wireless Emergency Alerts Can Help Impede the Spread of Covid-19

2021 ◽  
Vol 4 ◽  
pp. 33-36
Author(s):  
Hamilton Bean ◽  
Nels Grevstad ◽  
Alex Koutsoukos ◽  
Abigail Meyer
Keyword(s):  
United States ◽  
Statistical Analysis ◽  
Local Level ◽  
State Level ◽  
The United States ◽  
Small Sample ◽  
Effect Sizes ◽  
Clear Effect ◽  
Small Sample Sizes ◽  
The Right

This study offers a preliminary exploration of whether state-level (N=6) and local-level (N=53) Wireless Emergency Alert (WEA) messages might contribute to impeding the spread of Covid-19 in the United States. The study compares changes in reported rates of infections and deaths between states and localities that issued WEA messages in March and April of 2020 with states that did not. Small sample sizes and differences in the rates of Covid-19 spread prohibit robust statistical analysis and detection of clear effect sizes, but estimated effects are generally in the right direction.

scholarly journals Pension Accounting: Funding Costs And Liabilities Of Corporate Pension Plans

2011 ◽  
Vol 8 (2) ◽  
pp. 68
Author(s):  
Fawzi G. Dimian ◽  
Linda L. Kahlbaugh
Keyword(s):  
Statistical Analysis ◽  
Pension Plan ◽  
Pension Plans ◽  
Small Sample ◽  
Sample Sizes ◽  
Pension Accounting ◽  
Net Worth ◽  
Corporate Pension ◽  
Small Sample Sizes

Overall, pension plan assets analyzed in this study appear strong. They have excellent overall funding and unfunded vested liabilities would require less time to fund currently than in 1978. Pension expense per employee have been increasing, but at very nominal rates. And although the companies with the highest profits may not be the companies with the highest pension expenses, average pension expenses for most categories decreased. Currently unfunded vested liabilities are low relative to both pre-tax profits and net worth. Again, a number of points should be kept in mind when looking at these analysis and trends. Industry categories had small sample sizes. The sample sizes increase when companies are lumped into ranking categories making the data more representative. The overall trends include sample sizes of approximately 90, an acceptable number for statistical analysis. Also, some of the trends could be clouded by definitions of assets, liabilities, and income which differ from the 1978 study. However, after examining basic similarities between the studies and noting the strength of certain trends, the above mentioned conclusions appear warranted.

Sign in / Sign up

Export Citation Format

Share Document