Dichotomous thinking and informational waste in neuroimaging

Neuroimaging relies on separate statistical inferences at tens of thousands of spatial locations. Such massively univariate analysis typically requires adjustment for multiple testing in an attempt to maintain the family-wise error rate at a nominal level of 5%. We discuss how this approach is associated with substantial information loss because of an implicit but questionable assumption about the effect distribution across spatial units. To improve inference efficiency, predictive accuracy, and generalizability, we propose a Bayesian multilevel modeling framework. In addition, we make four actionable suggestions to alleviate information waste and to improve reproducibility: (1) abandon strict dichotomization; (2) report full results; (3) quantify effects, and (4) model data hierarchy.

Robust linear modelling in ElectroEncephaloGraphy can be obtained using single weights reflecting each single trials’ dynamics

Being able to remove or weigh down the influence of outlier data is desirable for any statistical models. While Magnetic and ElectroEncephaloGraphic (MEEG) data used to average trials per condition, it is now becoming common practice to use information from all trials to build linear models. Individual trials can, however, have considerable weight and thus bias inferential results. Here, rather than looking for outliers independently at each data point, we apply the principal component projection (PCP) method at each channel, deriving a single weight per trial at each channel independently. Using both synthetic data and open EEG data, we show (1) that PCP is efficient at detecting a large variety of outlying trials; (2) how PCP derived weights can be implemented in the context of the general linear model with accurate control of type 1 family-wise error rate; and (3) that our PCP-based Weighted Least Square (WLS) approach leads to in increase in power at the group results comparable to a much slower Iterative Reweighted Least Squares (IRLS), although the weighting scheme is markedly different. Together, results show that WLS based on PCP weights derived upon whole trial profiles is an efficient method to weigh down the influence of outlier data in linear models.Data availabilityall data used are publicly available (CC0), all code (simulations and data analyzes) is also available online in the LIMO MEEG GitHub repository (MIT license).

Covariate Adaptive Family-wise Error Rate Control for Genome-Wide Association Studies

The family-wise error rate (FWER) has been widely used in genome-wide association studies. With the increasing availability of functional genomics data, it is possible to increase the detection power by leveraging these genomic functional annotations. Previous efforts to accommodate covariates in multiple testing focus on the false discovery rate control while covariate-adaptive FWER-controlling procedures remain under-developed. Here we propose a novel covariate-adaptive FWER-controlling procedure that incorporates external covariates which are potentially informative of either the statistical power or the prior null probability. An efficient algorithm is developed to implement the proposed method. We prove its asymptotic validity and obtain the rate of convergence through a perturbation-type argument. Our numerical studies show that the new procedure is more powerful than competing methods and maintains robustness across different settings. We apply the proposed approach to the UK Biobank data and analyze 27 traits with 9 million single-nucleotide polymorphisms tested for associations. Seventy-five genomic annotations are used as covariates. Our approach detects more genome-wide significant loci than other methods in 21 out of the 27 traits.

A Structured Approach to Evaluating Life Course Hypotheses: Moving Beyond Analyses of Exposed Versus Unexposed in the Omics Context

The structured life course modeling approach (SLCMA) is a theory-driven analytic method that empirically compares multiple prespecified life course hypotheses characterizing time-dependent exposure-outcome relationships to determine which theory best fits the observed data. In this study, we performed simulations and empirical analyses to evaluate the performance of the SLCMA when applied to genome-wide DNA methylation (DNAm). Using simulations, we compared five statistical inference tests used with SLCMA (n=700), assessing the family-wise error rate, statistical power, and confidence interval coverage to determine whether inference based on these tests was valid in the presence of substantial multiple testing and small effects, two hallmark challenges of inference from omics data. In the empirical analyses, we evaluated the time-dependent relationship of childhood abuse with genome-wide DNAm (n=703). In simulations, selective inference and max-|t|-test performed best: both controlled family-wise error rate and yielded moderate statistical power. Empirical analyses using SLCMA revealed time-dependent effects of childhood abuse on DNAm. Our findings show that SLCMA, applied and interpreted appropriately, can be used in high-throughput settings to examine time-dependent effects underlying exposure-outcome relationships over the life course. We provide recommendations for applying the SLCMA in omics settings and encourage researchers to move beyond analyses of exposed versus unexposed.

Laminar analysis of the cortical T1/T2-weighted ratio at 7T

ObjectiveIn this observational study, we explored cortical structure as function of cortical depth through a laminar analysis of the T1/T2-weighted (T1w/T2w) ratio, which has been related to dendrite density in ex vivo brain tissue specimens of patients with MS.MethodsIn 39 patients (22 relapsing-remitting, 13 female, age 41.1 ± 10.6 years; 17 progressive, 11 female, age 54.1 ± 9.9 years) and 21 healthy controls (8 female, , age 41.6 ± 10.6 years), we performed a voxel-wise analysis of T1w/T2w ratio maps from high-resolution 7T images from the subpial surface to the gray matter/white matter boundary. Six layers were sampled to ensure accuracy based on mean cortical thickness and image resolution.ResultsAt the voxel-wise comparison (p < 0.05, family wise error rate corrected), the whole MS group showed lower T1w/T2w ratio values than controls, both when considering the entire cortex and each individual layer, with peaks occurring in the fusiform, temporo-occipital, and superior and middle frontal cortex. In relapsing-remitting patients, differences in the T1w/T2w ratio were only identified in the subpial layer, with the peak occurring in the fusiform cortex, whereas results obtained in progressive patients mirrored the widespread damage found in the whole group.ConclusionsLaminar analysis of T1w/T2w ratio mapping confirms the presence of cortical damage in MS and shows a variable expression of intracortical damage according to the disease phenotype. Although in the relapsing-remitting stage, only the subpial layer appears susceptible to damage, in progressive patients, widespread cortical abnormalities can be observed, not only, as described before, with regard to myelin/iron concentration but, possibly, to other microstructural features.

Distributions associated with simultaneous multiple hypothesis testing

We develop the distribution for the number of hypotheses found to be statistically significant using the rule from Simes (Biometrika 73: 751–754, 1986) for controlling the family-wise error rate (FWER). We find the distribution of the number of statistically significant p-values under the null hypothesis and show this follows a normal distribution under the alternative. We propose a parametric distribution ΨI(·) to model the marginal distribution of p-values sampled from a mixture of null uniform and non-uniform distributions under different alternative hypotheses. The ΨI distribution is useful when there are many different alternative hypotheses and these are not individually well understood. We fit ΨI to data from three cancer studies and use it to illustrate the distribution of the number of notable hypotheses observed in these examples. We model dependence in sampled p-values using a latent variable. These methods can be combined to illustrate a power analysis in planning a larger study on the basis of a smaller pilot experiment.

Erratum regarding “Optimizing effective numbers of tests by vine copula modeling”

We correct the definition of the family-wise error rate in our previous article “Optimizing effective numbers of tests by vine copula modeling”.