fMRI can be highly reliable, but it depends on what you measure

2020 ◽  
Author(s):  
Philip A. Kragel ◽  
Xiaochun Han ◽  
Thomas Edward Kraynak ◽  
Peter J. Gianaros ◽  
Tor D Wager
Keyword(s):  
Individual Differences ◽  
Region Of Interest ◽  
Small Sample ◽  
Measurement Models ◽  
Human Connectome Project ◽  
Show Evidence ◽  
Small Sample Sizes ◽  
Project Data ◽  
Test Retest Reliability ◽  
Task Types

Elliot and colleagues (2020) systematically evaluated the reliability of individual differences in task-based fMRI activity and found reliability to be poor. Here we demonstrate that task-based fMRI can be quite reliable, and that the small sample sizes, task types, and dated region of interest measures used in Elliot et al. lead to an overly negative picture. We show evidence from recent studies using multivariate models in larger samples, which have short-term test-retest reliability in the “excellent” range (ICC > 0.75). These include 8 fMRI studies of pain and a large study of affective images (N > 300). In addition, while some use cases for biomarkers require reliable individual differences, others do not. They require only that fMRI measures serve as reliable indicators of the presence of a mental state or event, which we term ‘task reliability’. In a re-analysis of the Human Connectome Project data reported in Elliot et al., we show excellent task reliability across roughly 4 months. Despite difficulties with some experimental paradigms and measurement models, the future is bright for fMRI research focused on biomarker development.

scholarly journals Including Data Analytical Stability in Cluster-based Inference

2019 ◽  
Author(s):  
Sanne P. Roels ◽  
Tom Loeys ◽  
Beatrijs Moerkerke
Keyword(s):  
Selection Process ◽  
Group Analysis ◽  
Testing Procedure ◽  
Small Sample ◽  
Sufficient Evidence ◽  
Analytical Stability ◽  
Human Connectome Project ◽  
Small Sample Sizes ◽  
Brain Data ◽  
Project Data

AbstractIn the statistical analysis of functional Magnetic Resonance Imaging (fMRI) brain data it remains a challenge to account for simultaneously testing activation in over 100.000 volume units or voxels. A popular method that reduces the dimensionality of this test problem is cluster-based inference. We propose a new testing procedure that allows to control the family-wise error (FWE) rate at the cluster level but improves cluster-based test decisions in two ways by (1) taking into account a measure for data analytical stability and (2) allowing a more voxel-based interpretation of the results. For each voxel, we define the re-selection rate conditional on a given FWE-corrected threshold and use this rate, which is a measure of stability, into the selection process. In our procedure, we set a more liberal and a more conservative FWE controlling threshold. Clusters that survive the liberal but not the conservative threshold are retained if sufficient evidence for voxelwise stability is available. Cluster that survive the conservative threshold are retained anyhow, and clusters that do not survive the liberal threshold are not further considered. Using the Human Connectome Project Data (Van Essen et al., 2012), we demonstrate how in a group analysis our method results not only in a higher number of selected voxels but also in a larger overlap between different test images. Additionally, we demonstrate the ability of our procedure to control the FWE, also in relatively small sample sizes.

scholarly journals Accurate predictions of individual differences in task-evoked brain activity from resting-state fMRI using a sparse ensemble learner

2021 ◽  
Author(s):  
Ying-Qiu Zheng ◽  
Seyedeh-Rezvan Farahibozorg ◽  
Weikang Gong ◽  
Hossein Rafipoor ◽  
Saad Jbabdi ◽  
...  
Keyword(s):  
Individual Differences ◽  
Resting State ◽  
Predictive Accuracy ◽  
Brain Activity ◽  
Resting State Fmri ◽  
Human Connectome Project ◽  
Wide Range ◽  
Dual Regression ◽  
Project Data ◽  
Test Retest Reliability

Modelling and predicting individual differences in task-evoked FMRI activity can have a wide range of applications from basic to clinical neuroscience. It has been shown that models based on resting-state activity can have high predictive accuracy. Here we propose several improvements to such models. Using a sparse ensemble leaner, we show that (i) features extracted using Stochastic Probabilistic Functional Modes (sPROFUMO) outperform the previously proposed dual-regression approach, (ii) that the shape and overall intensity of individualised task activations can be modelled separately and explicitly, (iii) training the model on predicting residual differences in brain activity further boosts individualised predictions. These results hold for both surface-based analyses of the Human Connectome Project data as well as volumetric analyses of UK-biobank data. Overall, our model achieves state of the art prediction accuracy on par with the test-retest reliability of tfMRI scans, suggesting that it has potential to supplement traditional task localisers.

scholarly journals What Role Does the Prefrontal Cortex Play in the Processing of Negative and Positive Stimuli in Adolescent Depression?

2019 ◽  
Vol 9 (5) ◽  
pp. 104 ◽  
Author(s):  
Siyabend Kaya ◽  
Ciara McCabe
Keyword(s):  
Prefrontal Cortex ◽  
Adolescent Depression ◽  
Region Of Interest ◽  
Small Sample ◽  
Symptoms Of Depression ◽  
Small Sample Sizes ◽  
Positive Stimuli ◽  
Larger Sample ◽  
Negative Stimuli

This perspective describes the contribution of the prefrontal cortex to the symptoms of depression in adolescents and specifically the processing of positive and negative information. We also discuss how the prefrontal cortex (PFC) activity and connectivity during tasks and at rest might be a biomarker for risk for depression onset in adolescents. We include some of our recent work examining not only the anticipation and consummation of positive and negative stimuli, but also effort to gain positive and avoid negative stimuli in adolescents with depression. We find, using region of interest analyses, that the PFC is blunted in those with depression compared to controls across the different phases but in a larger sample the PFC is blunted in the anticipatory phase of the study only. Taken together, in adolescents with depression there is evidence for dysfunctional PFC activity across different studies and tasks. However, the data are limited with small sample sizes and inconsistent findings. Larger longitudinal studies with more detailed assessments of symptoms across the spectrum are needed to further evaluate the role of the PFC in adolescent depression.

scholarly journals Integrating Task-Based Functional MRI Across Tasks Markedly Boosts Prediction and Reliability of Brain-Cognition Relationship

2021 ◽  
Author(s):  
Alina Tetereva ◽  
Jean Li ◽  
Jeremiah Deng ◽  
Argyris Stringaris ◽  
Narun Pat
Keyword(s):  
Individual Differences ◽  
Functional Mri ◽  
Cognitive Abilities ◽  
Brain Mri ◽  
Integration Theory ◽  
Sources Of Information ◽  
Resting State Functional Connectivity ◽  
Retest Reliability ◽  
Human Connectome Project ◽  
Test Retest Reliability

Capturing individual differences in cognitive abilities is central to human neuroscience. Yet our ability to estimate cognitive abilities via brain MRI is still poor in both prediction and reliability. Our study tested if this inability was partly due to the over-reliance on 1) non-task MRI modalities and 2) single modalities. We directly compared predictive models comprising of different sets of MRI modalities (e.g., task vs. non-task). Using the Human Connectome Project (n=873 humans, 473 females, after exclusions), we integrated task-based functional MRI (tfMRI) across seven tasks along with other non-task MRI modalities (structural MRI, resting-state functional connectivity) via a machine-learning, stacking approach. The model integrating all modalities provided unprecedented prediction (r=.581) and excellent test-retest reliability (ICC>.75) in capturing general cognitive abilities. Importantly, comparing to the model integrating among non-task modalities (r=.367), integrating tfMRI across tasks led to significantly higher prediction (r=.544) while still providing excellent test-retest reliability (ICC>.75). The model integrating tfMRI across tasks was driven by areas in the frontoparietal network and by tasks that are cognition-related (working-memory, relational processing, and language). This result is consistent with the parieto-frontal integration theory of intelligence. Accordingly, our results sharply contradict the recently popular notion that tfMRI is not appropriate for capturing individual differences in cognition. Instead, our study suggests that tfMRI, when used appropriately (i.e., by drawing information across the whole brain and across tasks and by integrating with other modalities), provides predictive and reliable sources of information for individual differences in cognitive abilities, more so than non-task modalities.

scholarly journals L2 (English) spelling is positively correlated with L2 vocabulary size, but not with a measure that putatively taps visual statistical learning

2019 ◽  
Author(s):  
Matthew HC Mak ◽  
Chen Qiu ◽  
Kathy K.M. Shum
Keyword(s):  
Individual Differences ◽  
Statistical Learning ◽  
Literacy Development ◽  
Small Sample ◽  
Sample Sizes ◽  
Vocabulary Size ◽  
Visual Statistical Learning ◽  
Small Sample Sizes ◽  
L2 Vocabulary ◽  
English Spelling

Recent studies showed that a person’s visual statistical learning (VSL) ability is positively correlated with L1 and L2 literacy development. That is, people who are more adept at detecting implicit regularities in visual inputs tend to be or to become better readers. Most if not all studies that examined this link looked only into reading ability, oddly neglecting spelling, and importantly, they all relied on relatively small sample sizes. In light of these, we recruited a relatively large sample (64 advanced English learners) to investigate if performance on a commonly adopted VSL task (i.e., embedded triplet task) correlates with L2 (English) spelling proficiency, as indexed by a spelling test involving highly rare regular English words. We also administered a rote memorization task and an L2 vocabulary size test. In line with a growing body of literature, we found no evidence that performance on the VSL task correlated with L2 spelling. We did however find clear evidence that L2 learners with a larger L2 vocabulary size are also better spellers. We argue that while SL is linked to literacy development, traditional VSL measures may not be suitable for the investigation of individual differences owing to poor sensitivity. Our study, alongside Mak (2016), also demonstrates that false positives may arise due to small sample sizes. We, therefore, urge researchers worldwide, especially those interested in individual differences, to work towards reproducibility and better science by for example avoiding unjustifiably small sample sizes.

scholarly journals Individual Cortical Entropy Profile: Test–Retest Reliability, Predictive Power for Cognitive Ability, and Neuroanatomical Foundation

2020 ◽  
Vol 1 (1) ◽  
Author(s):  
Mianxin Liu ◽  
Xinyang Liu ◽  
Andrea Hildebrandt ◽  
Changsong Zhou
Keyword(s):  
Individual Differences ◽  
Cognitive Ability ◽  
Cognitive Abilities ◽  
Predictive Power ◽  
Structural Connectivity ◽  
Cortical Folding ◽  
Retest Reliability ◽  
Human Connectome Project ◽  
Out Of Sample ◽  
Test Retest Reliability

Abstract The entropy profiles of cortical activity have become novel perspectives to investigate individual differences in behavior. However, previous studies have neglected foundational aspects of individual entropy profiles, that is, the test–retest reliability, the predictive power for cognitive ability in out-of-sample data, and the underlying neuroanatomical basis. We explored these issues in a large young healthy adult dataset (Human Connectome Project, N = 998). We showed the whole cortical entropy profile from resting-state functional magnetic resonance imaging is a robust personalized measure, while subsystem profiles exhibited heterogeneous reliabilities. The limbic network exhibited lowest reliability. We tested the out-of-sample predictive power for general and specific cognitive abilities based on reliable cortical entropy profiles. The default mode and visual networks are most crucial when predicting general cognitive ability. We investigated the anatomical features underlying cross-region and cross-individual variations in cortical entropy profiles. Cortical thickness and structural connectivity explained spatial variations in the group-averaged entropy profile. Cortical folding and myelination in the attention and frontoparietal networks determined predominantly individual cortical entropy profile. This study lays foundations for brain-entropy-based studies on individual differences to understand cognitive ability and related pathologies. These findings broaden our understanding of the associations between neural structures, functional dynamics, and cognitive ability.

Domain-General Models of Expertise

2019 ◽  
pp. 55-84 ◽  
Author(s):  
David Z. Hambrick ◽  
Alexander P. Burgoyne ◽  
Frederick L. Oswald
Keyword(s):  
Individual Differences ◽  
Job Performance ◽  
Cognitive Ability ◽  
Working Memory Capacity ◽  
Small Sample ◽  
Job Experience ◽  
Industrial Organizational Psychology ◽  
Small Sample Sizes ◽  
Restriction Of Range ◽  
Acquisition Of Expertise

This chapter reviews evidence concerning the contribution of cognitive ability to individual differences in expertise. The review covers research in traditional domains for expertise research such as music, sports, and chess, as well as research from industrial–organizational psychology on job performance. The specific question that we seek to address is whether domain-general measures of cognitive ability (e.g., IQ, working memory capacity, executive functioning, processing speed) predict individual differences in domain-relevant performance, especially beyond beginning levels of skill. Evidence from the expertise literature relevant to this question is difficult to interpret, due to small sample sizes, restriction of range, and other methodological limitations. By contrast, there is a wealth of consistent evidence that cognitive ability is a practically important and statistically significant predictor of job performance, even after extensive job experience. The chapter discusses ways that cognitive ability measures might be used in efforts to accelerate the acquisition of expertise.

Integrating Visual and Bayesian Statistical Analyses in Single Case Experimental Research to Evaluate the Effectiveness and Magnitude of a Comprehensive Behavioral Intervention

2018 ◽  
Author(s):  
Prathiba Natesan ◽  
Smita Mehta
Keyword(s):  
Behavioral Intervention ◽  
Effect Size ◽  
Rate Ratio ◽  
Visual Analysis ◽  
Single Case ◽  
Small Sample ◽  
Statistical Analyses ◽  
Data Types ◽  
Ratio Effect ◽  
Small Sample Sizes

Single case experimental designs (SCEDs) have become an indispensable methodology where randomized control trials may be impossible or even inappropriate. However, the nature of SCED data presents challenges for both visual and statistical analyses. Small sample sizes, autocorrelations, data types, and design types render many parametric statistical analyses and maximum likelihood approaches ineffective. The presence of autocorrelation decreases interrater reliability in visual analysis. The purpose of the present study is to demonstrate a newly developed model called the Bayesian unknown change-point (BUCP) model which overcomes all the above-mentioned data analytic challenges. This is the first study to formulate and demonstrate rate ratio effect size for autocorrelated data, which has remained an open question in SCED research until now. This expository study also compares and contrasts the results from BUCP model with visual analysis, and rate ratio effect size with nonoverlap of all pairs (NAP) effect size. Data from a comprehensive behavioral intervention are used for the demonstration.

Sign in / Sign up

Export Citation Format

Share Document