Shrinkage improves estimation of microbial associations under different normalization methods

Abstract Estimation of statistical associations in microbial genomic survey count data is fundamental to microbiome research. Experimental limitations, including count compositionality, low sample sizes and technical variability, obstruct standard application of association measures and require data normalization prior to statistical estimation. Here, we investigate the interplay between data normalization, microbial association estimation and available sample size by leveraging the large-scale American Gut Project (AGP) survey data. We analyze the statistical properties of two prominent linear association estimators, correlation and proportionality, under different sample scenarios and data normalization schemes, including RNA-seq analysis workflows and log-ratio transformations. We show that shrinkage estimation, a standard statistical regularization technique, can universally improve the quality of taxon–taxon association estimates for microbiome data. We find that large-scale association patterns in the AGP data can be grouped into five normalization-dependent classes. Using microbial association network construction and clustering as downstream data analysis examples, we show that variance-stabilizing and log-ratio approaches enable the most taxonomically and structurally coherent estimates. Taken together, the findings from our reproducible analysis workflow have important implications for microbiome studies in multiple stages of analysis, particularly when only small sample sizes are available.

Download Full-text

Shrinkage improves estimation of microbial associations under different normalization methods

10.1101/406264 ◽

2018 ◽

Cited By ~ 3

Author(s):

Michelle Badri ◽

Zachary D. Kurtz ◽

Richard Bonneau ◽

Christian L. Müller

Keyword(s):

Large Scale ◽

Standard Technique ◽

Small Sample ◽

Large Sample Size ◽

Data Normalization ◽

Sample Sizes ◽

Consistent Estimation ◽

Microbial Association ◽

Statistical Consistency ◽

Log Ratio

ABSTRACTConsistent estimation of associations in microbial genomic survey count data is fundamental to microbiome research. Technical limitations, including compositionality, low sample sizes, and technical variability, obstruct standard application of association measures and require data normalization prior to estimating associations. Here, we investigate the interplay between data normalization and microbial association estimation by a comprehensive analysis of statistical consistency. Leveraging the large sample size of the American Gut Project (AGP), we assess the consistency of the two prominent linear association estimators, correlation and proportionality, under different sample scenarios and data normalization schemes, including RNA-seq analysis work flows and log-ratio transformations. We show that shrinkage estimation, a standard technique in high-dimensional statistics, can universally improve the quality of association estimates for microbiome data. We find that large-scale association patterns in the AGP data can be grouped into five normalization-dependent classes. Using microbial association network construction and clustering as examples of exploratory data analysis, we show that variance-stabilizing and log-ratio approaches provide for the most consistent estimation of taxonomic and structural coherence. Taken together, the findings from our reproducible analysis workflow have important implications for microbiome studies in multiple stages of analysis, particularly when only small sample sizes are available.

Download Full-text

Applied comparison of large‐scale propensity score matching and cardinality matching for causal inference in observational research

BMC Medical Research Methodology ◽

10.1186/s12874-021-01282-1 ◽

2021 ◽

Vol 21 (1) ◽

Author(s):

Stephen P. Fortin ◽

Stephen S Johnston ◽

Martijn J Schuemie

Keyword(s):

Large Scale ◽

Negative Control ◽

Small Sample ◽

Matched Sample ◽

Sample Sizes ◽

Covariate Balance ◽

Full Study ◽

Study Population ◽

Small Sample Sizes ◽

Residual Confounding

Abstract Background Cardinality matching (CM), a novel matching technique, finds the largest matched sample meeting prespecified balance criteria thereby overcoming limitations of propensity score matching (PSM) associated with limited covariate overlap, which are especially pronounced in studies with small sample sizes. The current study proposes a framework for large-scale CM (LS-CM); and compares large-scale PSM (LS-PSM) and LS-CM in terms of post-match sample size, covariate balance and residual confounding at progressively smaller sample sizes. Methods Evaluation of LS-PSM and LS-CM within a comparative cohort study of new users of angiotensin-converting enzyme inhibitor (ACEI) and thiazide or thiazide-like diuretic monotherapy identified from a U.S. insurance claims database. Candidate covariates included patient demographics, and all observed prior conditions, drug exposures and procedures. Propensity scores were calculated using LASSO regression, and candidate covariates with non-zero beta coefficients in the propensity model were defined as matching covariates for use in LS-CM. One-to-one matching was performed using progressively tighter parameter settings. Covariate balance was assessed using standardized mean differences. Hazard ratios for negative control outcomes perceived as unassociated with treatment (i.e., true hazard ratio of 1) were estimated using unconditional Cox models. Residual confounding was assessed using the expected systematic error of the empirical null distribution of negative control effect estimates compared to the ground truth. To simulate diverse research conditions, analyses were repeated within 10 %, 1 and 0.5 % subsample groups with increasingly limited covariate overlap. Results A total of 172,117 patients (ACEI: 129,078; thiazide: 43,039) met the study criteria. As compared to LS-PSM, LS-CM was associated with increased sample retention. Although LS-PSM achieved balance across all matching covariates within the full study population, substantial matching covariate imbalance was observed within the 1 and 0.5 % subsample groups. Meanwhile, LS-CM achieved matching covariate balance across all analyses. LS-PSM was associated with better candidate covariate balance within the full study population. Otherwise, both matching techniques achieved comparable candidate covariate balance and expected systematic error. Conclusions LS-CM found the largest matched sample meeting prespecified balance criteria while achieving comparable candidate covariate balance and residual confounding. We recommend LS-CM as an alternative to LS-PSM in studies with small sample sizes or limited covariate overlap.

Download Full-text

Crowdsourcing in Cognitive and Systems Neuroscience

The Neuroscientist ◽

10.1177/10738584211017018 ◽

2021 ◽

pp. 107385842110170

Author(s):

Brian P. Johnson ◽

Eran Dayan ◽

Nitzan Censor ◽

Leonardo G. Cohen

Keyword(s):

Large Scale ◽

Eye Gaze ◽

Real Life ◽

Research Participant ◽

Small Sample ◽

Sample Sizes ◽

Motor Tasks ◽

Systems Neuroscience ◽

Small Sample Sizes ◽

Human Systems

Behavioral research in cognitive and human systems neuroscience has been largely carried out in-person in laboratory settings. Underpowering and lack of reproducibility due to small sample sizes have weakened conclusions of these investigations. In other disciplines, such as neuroeconomics and social sciences, crowdsourcing has been extensively utilized as a data collection tool, and a means to increase sample sizes. Recent methodological advances allow scientists, for the first time, to test online more complex cognitive, perceptual, and motor tasks. Here we review the nascent literature on the use of online crowdsourcing in cognitive and human systems neuroscience. These investigations take advantage of the ability to reliably track the activity of a participant’s computer keyboard, mouse, and eye gaze in the context of large-scale studies online that involve diverse research participant pools. Crowdsourcing allows for testing the generalizability of behavioral hypotheses in real-life environments that are less accessible to lab-designed investigations. Crowdsourcing is further useful when in-laboratory studies are limited, for example during the current COVID-19 pandemic. We also discuss current limitations of crowdsourcing research, and suggest pathways to address them. We conclude that online crowdsourcing is likely to widen the scope and strengthen conclusions of cognitive and human systems neuroscience investigations.

Download Full-text

Problems with small sample sizes in psychophysiological research

PsycEXTRA Dataset ◽

10.1037/e526132012-267 ◽

1996 ◽

Author(s):

Todd C. Riniolo ◽

Stephen W. Porges

Keyword(s):

Small Sample ◽

Sample Sizes ◽

Psychophysiological Research ◽

Small Sample Sizes

Download Full-text

Bayesian Latent Growth Mixture-Modeling With Small Sample Sizes

PsycEXTRA Dataset ◽

10.1037/e568142014-001 ◽

2014 ◽

Author(s):

Sarah Depaoli

Keyword(s):

Growth Mixture Modeling ◽

Mixture Modeling ◽

Small Sample ◽

Sample Sizes ◽

Latent Growth ◽

Growth Mixture ◽

Latent Growth Mixture Modeling ◽

Small Sample Sizes

Download Full-text

No Evidence that Experiencing Physical Warmth Promotes Interpersonal Warmth: Two Failures to Replicate Williams and Bargh (2008)

10.31234/osf.io/mvn9b ◽

2018 ◽

Cited By ~ 1

Author(s):

Christopher Chabris ◽

Patrick Ryan Heck ◽

Jaclyn Mandart ◽

Daniel Jacob Benjamin ◽

Daniel J. Simons

Keyword(s):

Null Hypothesis ◽

Small Sample ◽

Sample Sizes ◽

Double Blind ◽

Bayesian Analyses ◽

Physical Warmth ◽

Small Sample Sizes ◽

Interpersonal Warmth

Williams and Bargh (2008) reported that holding a hot cup of coffee caused participants to judge a person’s personality as warmer, and that holding a therapeutic heat pad caused participants to choose rewards for other people rather than for themselves. These experiments featured large effects (r = .28 and .31), small sample sizes (41 and 53 participants), and barely statistically significant results. We attempted to replicate both experiments in field settings with more than triple the sample sizes (128 and 177) and double-blind procedures, but found near-zero effects (r = –.03 and .02). In both cases, Bayesian analyses suggest there is substantially more evidence for the null hypothesis of no effect than for the original physical warmth priming hypothesis.

Download Full-text

Use of Antiplatelet Agents Decreases the Positive Predictive Value of Fecal Immunochemical Tests for Colorectal Cancer but Does Not Affect Their Sensitivity

Journal of Personalized Medicine ◽

10.3390/jpm11060497 ◽

2021 ◽

Vol 11 (6) ◽

pp. 497

Author(s):

Yoonsuk Jung ◽

Eui Im ◽

Jinhee Lee ◽

Hyeah Lee ◽

Changmo Moon

Keyword(s):

Colorectal Cancer ◽

Large Scale ◽

Screening Program ◽

Antiplatelet Agents ◽

Population Based ◽

Small Sample ◽

Antithrombotic Agents ◽

Predictive Values ◽

Prescription Rates ◽

Small Sample Sizes

Previous studies have evaluated the effects of antithrombotic agents on the performance of fecal immunochemical tests (FITs) for the detection of colorectal cancer (CRC), but the results were inconsistent and based on small sample sizes. We studied this topic using a large-scale population-based database. Using the Korean National Cancer Screening Program Database, we compared the performance of FITs for CRC detection between users and non-users of antiplatelet agents and warfarin. Non-users were matched according to age and sex. Among 5,426,469 eligible participants, 768,733 used antiplatelet agents (mono/dual/triple therapy, n = 701,683/63,211/3839), and 19,569 used warfarin, while 4,638,167 were non-users. Among antiplatelet agents, aspirin, clopidogrel, and cilostazol ranked first, second, and third, respectively, in terms of prescription rates. Users of antiplatelet agents (3.62% vs. 4.45%; relative risk (RR): 0.83; 95% confidence interval (CI): 0.78–0.88), aspirin (3.66% vs. 4.13%; RR: 0.90; 95% CI: 0.83–0.97), and clopidogrel (3.48% vs. 4.88%; RR: 0.72; 95% CI: 0.61–0.86) had lower positive predictive values (PPVs) for CRC detection than non-users. However, there were no significant differences in PPV between cilostazol vs. non-users and warfarin users vs. non-users. For PPV, the RR (users vs. non-users) for antiplatelet monotherapy was 0.86, while the RRs for dual and triple antiplatelet therapies (excluding cilostazol) were 0.67 and 0.22, respectively. For all antithrombotic agents, the sensitivity for CRC detection was not different between users and non-users. Use of antiplatelet agents, except cilostazol, may increase the false positives without improving the sensitivity of FITs for CRC detection.

Download Full-text

G-computation and machine learning for estimating the causal effects of binary exposure statuses on binary outcomes

Scientific Reports ◽

10.1038/s41598-021-81110-0 ◽

2021 ◽

Vol 11 (1) ◽

Author(s):

Florent Le Borgne ◽

Arthur Chatton ◽

Maxime Léger ◽

Rémi Lenain ◽

Yohann Foucher

Keyword(s):

Machine Learning ◽

Support Vector Machine ◽

Statistical Power ◽

Small Sample ◽

Causal Effects ◽

Small Samples ◽

Support Vector ◽

Sample Sizes ◽

Super Learner ◽

Small Sample Sizes

AbstractIn clinical research, there is a growing interest in the use of propensity score-based methods to estimate causal effects. G-computation is an alternative because of its high statistical power. Machine learning is also increasingly used because of its possible robustness to model misspecification. In this paper, we aimed to propose an approach that combines machine learning and G-computation when both the outcome and the exposure status are binary and is able to deal with small samples. We evaluated the performances of several methods, including penalized logistic regressions, a neural network, a support vector machine, boosted classification and regression trees, and a super learner through simulations. We proposed six different scenarios characterised by various sample sizes, numbers of covariates and relationships between covariates, exposure statuses, and outcomes. We have also illustrated the application of these methods, in which they were used to estimate the efficacy of barbiturates prescribed during the first 24 h of an episode of intracranial hypertension. In the context of GC, for estimating the individual outcome probabilities in two counterfactual worlds, we reported that the super learner tended to outperform the other approaches in terms of both bias and variance, especially for small sample sizes. The support vector machine performed well, but its mean bias was slightly higher than that of the super learner. In the investigated scenarios, G-computation associated with the super learner was a performant method for drawing causal inferences, even from small sample sizes.

Download Full-text