scholarly journals Handling Multiplicity in Neuroimaging through Bayesian Lenses with Multilevel Modeling

2017 ◽  
Author(s):  
Gang Chen ◽  
Yaqiong Xiao ◽  
Paul A. Taylor ◽  
Justin K. Rajendra ◽  
Tracy Riggins ◽  
...  
Keyword(s):  
Multiple Testing ◽  
False Positive Rate ◽  
Region Of Interest ◽  
Brain Regions ◽  
Integrative Model ◽  
Multidimensional Data ◽  
Positive Rate ◽  
Incorrect Sign ◽  
Sign Type ◽  
Univariate Approach

AbstractHere we address the current issues of inefficiency and over-penalization in the massively univariate approach followed by the correction for multiple testing, and propose a more efficient model that pools and shares information among brain regions. Using Bayesian multilevel (BML) modeling, we control two types of error that are more relevant than the conventional false positive rate (FPR): incorrect sign (type S) and incorrect magnitude (type M). BML also aims to achieve two goals: 1) improving modeling efficiency by having one integrative model and thereby dissolving the multiple testing issue, and 2) turning the focus of conventional null hypothesis significant testing (NHST) on FPR into quality control by calibrating type S errors while maintaining a reasonable level of inference efficiency The performance and validity of this approach are demonstrated through an application at the region of interest (ROI) level, with all the regions on an equal footing: unlike the current approaches under NHST, small regions are not disadvantaged simply because of their physical size. In addition, compared to the massively univariate approach, BML may simultaneously achieve increased spatial specificity and inference efficiency, and promote results reporting in totality and transparency. The benefits of BML are illustrated in performance and quality checking using an experimental dataset. The methodology also avoids the current practice of sharp and arbitrary thresholding in thep-value funnel to which the multidimensional data are reduced. The BML approach with its auxiliary tools is available as part of the AFNI suite for general use.

scholarly journals All-Resolutions Inference for Brain Imaging

2017 ◽  
Author(s):  
Jonathan D. Rosenblatt ◽  
Livio Finos ◽  
Wouter D. Weeda ◽  
Aldo Solari ◽  
Jelle J. Goeman
Keyword(s):  
Random Field ◽  
Multiple Testing ◽  
False Positive Rate ◽  
Brain Regions ◽  
Evoked Response ◽  
Recent Criticism ◽  
Random Field Theory ◽  
Positive Rate ◽  
Multiple Clusters ◽  
Spatial Specificity

AbstractThe most prevalent approach to activation localization in neuroimaging is to identify brain regions as contiguous supra-threshold clusters, check their significance using random field theory, and correct for the multiple clusters being tested. Besides recent criticism on the validity of the random field assumption, a spatial specificity paradox remains: the larger the detected cluster, the less we know about the location of activation within that cluster. This is because cluster inference implies “there exists at least one voxel with an evoked response in the cluster”, and not that “all the voxels in the cluster have an evoked response”. Inference on voxels within selected clusters is considered bad practice, due to the voxel-wise false positive rate inflation associated with this circular inference. Here, we propose a remedy to the spatial specificity paradox. By applying recent results from the multiple testing statistical literature, we are able to quantify the proportion of truly active voxels within selected clusters, an approach we call All-Resolutions Inference (ARI). If this proportion is high, the paradox vanishes. If it is low, we can further “drill down” from the cluster level to sub-regions, and even to individual voxels, in order to pinpoint the origin of the activation. In fact, ARI allows inference on the proportion of activation in all voxel sets, no matter how large or small, however these have been selected, all from the same data. We use two fMRI datasets to demonstrate the non-triviality of the spatial specificity paradox, and its resolution using ARI. One of these datasets is large enough for us to split it and validate the ARI estimates. The conservatism of ARI inference permits circularity without losing error guarantees, while still returning informative estimates.

Considerations on the region of interest in the ROC space

2021 ◽  
pp. 096228022110605
Author(s):  
Luigi Lavazza ◽  
Sandro Morasca
Keyword(s):  
Receiver Operating Characteristic ◽  
Operating Characteristic ◽  
False Positive Rate ◽  
Area Under The Curve ◽  
Region Of Interest ◽  
True Positive Rate ◽  
True Positive ◽  
Receiver Operating Characteristic Curves ◽  
Positive Rate ◽  
Receiver Operating

Receiver Operating Characteristic curves have been widely used to represent the performance of diagnostic tests. The corresponding area under the curve, widely used to evaluate their performance quantitatively, has been criticized in several respects. Several proposals have been introduced to improve area under the curve by taking into account only specific regions of the Receiver Operating Characteristic space, that is, the plane to which Receiver Operating Characteristic curves belong. For instance, a region of interest can be delimited by setting specific thresholds for the true positive rate or the false positive rate. Different ways of setting the borders of the region of interest may result in completely different, even opposing, evaluations. In this paper, we present a method to define a region of interest in a rigorous and objective way, and compute a partial area under the curve that can be used to evaluate the performance of diagnostic tests. The method was originally conceived in the Software Engineering domain to evaluate the performance of methods that estimate the defectiveness of software modules. We compare this method with previous proposals. Our method allows the definition of regions of interest by setting acceptability thresholds on any kind of performance metric, and not just false positive rate and true positive rate: for instance, the region of interest can be determined by imposing that [Formula: see text] (also known as the Matthews Correlation Coefficient) is above a given threshold. We also show how to delimit the region of interest corresponding to acceptable costs, whenever the individual cost of false positives and false negatives is known. Finally, we demonstrate the effectiveness of the method by applying it to the Wisconsin Breast Cancer Data. We provide Python and R packages supporting the presented method.

scholarly journals A parameter-free statistical test that improves the detection of neuronal responsiveness

2020 ◽  
Author(s):  
Jorrit S. Montijn ◽  
Koen Seignette ◽  
Marcus H. Howlett ◽  
J. Leonie Cazemier ◽  
Maarten Kamermans ◽  
...  
Keyword(s):  
False Positive Rate ◽  
Spatial Location ◽  
Brain Regions ◽  
Arbitrary Parameter ◽  
Manual Curation ◽  
Neurophysiological Studies ◽  
Positive Rate ◽  
Reliable Quantification ◽  
Stimulation Current ◽  
Recording Techniques

AbstractNeurophysiological studies depend on a reliable quantification of whether and when a neuron responds to stimulation. Current methods to determine responsiveness require arbitrary parameter choices, such as binning size. These choices can change the results, which invites bad statistical practice and reduces the replicability. Moreover, many methods only detect mean-rate modulated cells. New recording techniques that yield increasingly large numbers of cells would benefit from a test for cell-inclusion that requires no manual curation. Here, we present the parameter-free ZETA-test, which outperforms t-tests and ANOVAs by including more cells at a similar false-positive rate. We show that our procedure works across brain regions and recording techniques, including calcium imaging and Neuropixels data. Furthermore, in illustration of the method, we show in mouse visual cortex that 1) visuomotor-mismatch and spatial location are encoded by different neuronal subpopulations; and 2) optogenetic stimulation of VIP cells leads to early inhibition and subsequent disinhibition.

scholarly journals A parameter-free statistical test for neuronal responsiveness

eLife ◽  
2021 ◽  
Vol 10 ◽  
Author(s):  
Jorrit Steven Montijn ◽  
Koen Seignette ◽  
Marcus H Howlett ◽  
J Leonie Cazemier ◽  
Maarten Kamermans ◽  
...  
Keyword(s):  
False Positive Rate ◽  
Spatial Location ◽  
Brain Regions ◽  
Arbitrary Parameter ◽  
Manual Curation ◽  
Neurophysiological Studies ◽  
Positive Rate ◽  
Reliable Quantification ◽  
Mouse Visual Cortex ◽  
Recording Techniques

Neurophysiological studies depend on a reliable quantification of whether and when a neuron responds to stimulation. Simple methods to determine responsiveness require arbitrary parameter choices, such as binning size, while more advanced model-based methods require fitting and hyperparameter tuning. These parameter choices can change the results, which invites bad statistical practice and reduces the replicability. New recording techniques that yield increasingly large numbers of cells would benefit from a test for cell-inclusion that requires no manual curation. Here, we present the parameter-free ZETA-test, which outperforms t-tests, ANOVAs, and renewal-process-based methods by including more cells at a similar false-positive rate. We show that our procedure works across brain regions and recording techniques, including calcium imaging and Neuropixels data. Furthermore, in illustration of the method, we show in mouse visual cortex that 1) visuomotor-mismatch and spatial location are encoded by different neuronal subpopulations; and 2) optogenetic stimulation of VIP cells leads to early inhibition and subsequent disinhibition.

scholarly journals Fully Automated Support System for Diagnosis of Breast Cancer in Contrast-Enhanced Spectral Mammography Images

2019 ◽  
Vol 8 (6) ◽  
pp. 891 ◽  
Author(s):  
Annarita Fanizzi ◽  
Liliana Losurdo ◽  
Teresa Maria A. Basile ◽  
Roberto Bellotti ◽  
Ubaldo Bottigli ◽  
...  
Keyword(s):  
Breast Cancer ◽  
False Positive Rate ◽  
Region Of Interest ◽  
Classification Model ◽  
Support Tool ◽  
Contrast Enhanced ◽  
Positive Rate ◽  
Sequential Feature Selection ◽  
Fully Automatic ◽  
Automated Support

Contrast-Enhanced Spectral Mammography (CESM) is a novelty instrumentation for diagnosing of breast cancer, but it can still be considered operator dependent. In this paper, we proposed a fully automatic system as a diagnostic support tool for the clinicians. For each Region Of Interest (ROI), a features set was extracted from low-energy and recombined images by using different techniques. A Random Forest classifier was trained on a selected subset of significant features by a sequential feature selection algorithm. The proposed Computer-Automated Diagnosis system is tested on 48 ROIs extracted from 53 patients referred to Istituto Tumori “Giovanni Paolo II” of Bari (Italy) from the breast cancer screening phase between March 2017 and June 2018. The present method resulted highly performing in the prediction of benign/malignant ROIs with median values of sensitivity and specificity of 87 . 5 % and 91 . 7 % , respectively. The performance was high compared to the state-of-the-art, even with a moderate/marked level of parenchymal background. Our classification model outperformed the human reader, by increasing the specificity over 8 % . Therefore, our system could represent a valid support tool for radiologists for interpreting CESM images, both reducing the false positive rate and limiting biopsies and surgeries.

Improving diagnosis of acute appendicitis with atypical findings by Tc-99m HMPAO leukocyte scan

2002 ◽  
Vol 41 (01) ◽  
pp. 37-41 ◽  
Author(s):  
S. Shung-Shung ◽  
S. Yu-Chien ◽  
Y. Mei-Due ◽  
W. Hwei-Chung ◽  
A. Kao
Keyword(s):  
Acute Appendicitis ◽  
False Positive ◽  
False Positive Rate ◽  
Accurate Method ◽  
Clinical Findings ◽  
Pathological Findings ◽  
Lower Quadrant ◽  
Predictive Values ◽  
Positive Rate

Summary Aim: Even with careful observation, the overall false-positive rate of laparotomy remains 10-15% when acute appendicitis was suspected. Therefore, the clinical efficacy of Tc-99m HMPAO labeled leukocyte (TC-WBC) scan for the diagnosis of acute appendicitis in patients presenting with atypical clinical findings is assessed. Patients and Methods: Eighty patients presenting with acute abdominal pain and possible acute appendicitis but atypical findings were included in this study. After intravenous injection of TC-WBC, serial anterior abdominal/pelvic images at 30, 60, 120 and 240 min with 800k counts were obtained with a gamma camera. Any abnormal localization of radioactivity in the right lower quadrant of the abdomen, equal to or greater than bone marrow activity, was considered as a positive scan. Results: 36 out of 49 patients showing positive TC-WBC scans received appendectomy. They all proved to have positive pathological findings. Five positive TC-WBC were not related to acute appendicitis, because of other pathological lesions. Eight patients were not operated and clinical follow-up after one month revealed no acute abdominal condition. Three of 31 patients with negative TC-WBC scans received appendectomy. They also presented positive pathological findings. The remaining 28 patients did not receive operations and revealed no evidence of appendicitis after at least one month of follow-up. The overall sensitivity, specificity, accuracy, positive and negative predictive values for TC-WBC scan to diagnose acute appendicitis were 92, 78, 86, 82, and 90%, respectively. Conclusion: TC-WBC scan provides a rapid and highly accurate method for the diagnosis of acute appendicitis in patients with equivocal clinical examination. It proved useful in reducing the false-positive rate of laparotomy and shortens the time necessary for clinical observation.

Predicting Fetal Chromosome Anomalies in the First Trimester Using Pregnancy Associated Plasma Protein-A: A Comparison of Statistical Methods

1993 ◽  
Vol 32 (02) ◽  
pp. 175-179 ◽  
Author(s):  
B. Brambati ◽  
T. Chard ◽  
J. G. Grudzinskas ◽  
M. C. M. Macintosh
Keyword(s):  
Logistic Regression ◽  
General Population ◽  
Likelihood Ratio ◽  
False Positive ◽  
False Positive Rate ◽  
Ratio Method ◽  
Detection Rates ◽  
Gaussian Distributions ◽  
Positive Rate ◽  
Likelihood Ratio Method

Abstract:The analysis of the clinical efficiency of a biochemical parameter in the prediction of chromosome anomalies is described, using a database of 475 cases including 30 abnormalities. A comparison was made of two different approaches to the statistical analysis: the use of Gaussian frequency distributions and likelihood ratios, and logistic regression. Both methods computed that for a 5% false-positive rate approximately 60% of anomalies are detected on the basis of maternal age and serum PAPP-A. The logistic regression analysis is appropriate where the outcome variable (chromosome anomaly) is binary and the detection rates refer to the original data only. The likelihood ratio method is used to predict the outcome in the general population. The latter method depends on the data or some transformation of the data fitting a known frequency distribution (Gaussian in this case). The precision of the predicted detection rates is limited by the small sample of abnormals (30 cases). Varying the means and standard deviations (to the limits of their 95% confidence intervals) of the fitted log Gaussian distributions resulted in a detection rate varying between 42% and 79% for a 5% false-positive rate. Thus, although the likelihood ratio method is potentially the better method in determining the usefulness of a test in the general population, larger numbers of abnormal cases are required to stabilise the means and standard deviations of the fitted log Gaussian distributions.

Identification of and Correction for Publication Bias: Comment

2019 ◽  
Author(s):  
Amanda Kvarven ◽  
Eirik Strømland ◽  
Magnus Johannesson
Keyword(s):  
Publication Bias ◽  
False Positive ◽  
Large Scale ◽  
Meta Analysis ◽  
False Positive Rate ◽  
Effect Sizes ◽  
Replication Studies ◽  
Moderate Reduction ◽  
Positive Rate ◽  
Meta Analyses

Andrews & Kasy (2019) propose an approach for adjusting effect sizes in meta-analysis for publication bias. We use the Andrews-Kasy estimator to adjust the result of 15 meta-analyses and compare the adjusted results to 15 large-scale multiple labs replication studies estimating the same effects. The pre-registered replications provide precisely estimated effect sizes, which do not suffer from publication bias. The Andrews-Kasy approach leads to a moderate reduction of the inflated effect sizes in the meta-analyses. However, the approach still overestimates effect sizes by a factor of about two or more and has an estimated false positive rate of between 57% and 100%.

Forms, Importance, and Ineffability of Factor Interactions to Define Personality Disorders: Comment on Lilienfeld et al. (2019)

2019 ◽  
Author(s):  
Stephen D Benning ◽  
Edward Smith
Keyword(s):  
Personality Disorders ◽  
False Positive Rate ◽  
Empirical Work ◽  
Design Studies ◽  
Linear Threshold ◽  
Positive Rate ◽  
Saturation Effects ◽  
Factor Interactions ◽  
Main Effects ◽  
Criterion Variables

The emergent interpersonal syndrome (EIS) approach conceptualizes personality disorders as the interaction among their constituent traits to predict important criterion variables. We detail the difficulties we have experienced finding such interactive predictors in our empirical work on psychopathy, even when using uncorrelated traits that maximize power. Rather than explaining a large absolute proportion of variance in interpersonal outcomes, EIS interactions might explain small amounts of variance relative to the main effects of each trait. Indeed, these interactions may necessitate samples of almost 1,000 observations for 80% power and a false positive rate of .05. EIS models must describe which specific traits’ interactions constitute a particular EIS, as effect sizes appear to diminish as higher-order trait interactions are analyzed. Considering whether EIS interactions are ordinal with non-crossing slopes, disordinal with crossing slopes, or entail non-linear threshold or saturation effects may help researchers design studies, sampling strategies, and analyses to model their expected effects efficiently.

The false positive rate of P300-based concealed information test

2018 ◽  
Vol 30 (3) ◽  
pp. 241-259 ◽  
Author(s):  
엄진섭 ◽  
Jin-Hun Sohn ◽  
Hajung Jeon
Keyword(s):  
False Positive ◽  
False Positive Rate ◽  
Concealed Information Test ◽  
Positive Rate ◽  
Concealed Information ◽  
Information Test
Sign in / Sign up

Export Citation Format

Share Document