Marginal Effects in Interaction Models: Determining and Controlling the False Positive Rate

2017 ◽  
Vol 51 (9) ◽  
pp. 1144-1176 ◽  
Author(s):  
Justin Esarey ◽  
Jane Lawrence Sumner
Keyword(s):  
Political Parties ◽  
False Positive ◽  
Software Package ◽  
Statistical Significance ◽  
False Positive Rate ◽  
Marginal Effect ◽  
Marginal Effects ◽  
Political Studies ◽  
Positive Rate ◽  
The Relationship

When a researcher suspects that the marginal effect of [Formula: see text] on [Formula: see text] varies with [Formula: see text], a common approach is to plot [Formula: see text] at different values of [Formula: see text] along with a pointwise confidence interval generated using the procedure described in Brambor, Clark, and Golder to assess the magnitude and statistical significance of the relationship. Our article makes three contributions. First, we demonstrate that the Brambor, Clark, and Golder approach produces statistically significant findings when [Formula: see text] at a rate that can be many times larger or smaller than the nominal false positive rate of the test. Second, we introduce the interactionTest software package for R to implement procedures that allow easy control of the false positive rate. Finally, we illustrate our findings by replicating an empirical analysis of the relationship between ethnic heterogeneity and the number of political parties from Comparative Political Studies.

The performance of delta check methods.

1979 ◽  
Vol 25 (12) ◽  
pp. 2034-2037 ◽  
Author(s):  
L B Sheiner ◽  
L A Wheeler ◽  
J K Moore
Keyword(s):  
False Positive ◽  
False Positive Rate ◽  
True Positive Rate ◽  
True Positive ◽  
Discriminant Functions ◽  
Linear Discriminant ◽  
Operating Rate ◽  
Positive Rate ◽  
The Relationship

Abstract The percentage of mislabeled specimens detected (true-positive rate) and the percentage of correctly labeled specimens misidentified (false-positive rate) were computed for three previously proposed delta check methods and two linear discriminant functions. The true-positive rate was computed from a set of pairs of specimens, each having one member replaced by a member from another pair chosen at random. The relationship between true-positive and false-positive rates was similar among the delta check methods tested, indicating equal performance for all of them over the range of false-positive rate of interest. At a practical false-positive operating rate of about 5%, delta check methods detect only about 50% of mislabeled specimens; even if the actual mislabeling rate is moderate (e.g., 1%), only abot 10% of specimens flagged a by a delta check will actually have been mislabeled.

scholarly journals The impact of P-hacking on "Redefine statistical significance"

2017 ◽  
Author(s):  
Harry Crane
Keyword(s):  
False Positive ◽  
Statistical Significance ◽  
False Positive Rate ◽  
Perceived Benefits ◽  
Replication Rate ◽  
Recent Proposal ◽  
Positive Rate ◽  
Replication Crisis ◽  
The Impact ◽  
Lower Cutoff

A recent proposal to "redefine statistical significance" (Benjamin, et al. Nature Human Behaviour, 2017) claims that false positive rates "would immediately improve" by factors greater than two and replication rates would double simply by changing the conventional cutoff for 'statistical significance' from P<0.05 to P<0.005. I analyze the veracity of these claims, focusing especially on how Benjamin, et al neglect the effects of P-hacking in assessing the impact of their proposal. My analysis shows that once P-hacking is accounted for the perceived benefits of the lower threshold all but disappear, prompting two main conclusions: (i) The claimed improvements to false positive rate and replication rate in Benjamin, et al (2017) are exaggerated and misleading. (ii) There are plausible scenarios under which the lower cutoff will make the replication crisis worse.

scholarly journals Inferring the genetic architecture of expression variation from replicated high throughput allele-specific expression experiments

2019 ◽  
Author(s):  
Xinwen Zhang ◽  
J.J. Emerson
Keyword(s):  
False Positive ◽  
Statistical Significance ◽  
False Positive Rate ◽  
Regulatory Sequences ◽  
Binomial Model ◽  
Specific Expression ◽  
Allele Specific Expression ◽  
Expression Variation ◽  
Positive Rate ◽  
Allele Specific

AbstractGene expression variation between alleles in a diploid cell is mediated by variation in cis regulatory sequences, which usually refers to the differences in DNA sequence between two alleles near the gene of interest. Expression differences caused by cis variation has been estimated by the ratio of the expression level of the two alleles under a binomial model. However, the binomial model underestimates the variance among replicated experiments resulting in the exaggerated statistical significance of estimated cis effects and thus many false discoveries of cis-affected genes. Here we describe a beta-binomial model that estimates the cis-effect for each gene while permitting overdispersion of variance among replicates. We demonstrated with simulated null data (data without true cis-effect) that the new model fits the true distribution better, resulting in approximately 5% false positive rate under 5% significance level in all null datasets, considerably better than the 6%-40% false positive rate of the binomial model. Additional replicates increase the performance of the beta-binomial model but not of the binomial model. We also collected new allele-specific expression data from an experiment comprised of 20 replicates of a yeast hybrid (YPS128/RM11-1a). We eliminated the mapping bias problem with de novo assemblies of the two parental genomes. By applying the beta-binomial model to this dataset, we found that cis effects are ubiquitous, affecting around 70% of genes. However, most of these changes are small in magnitude. The high number of replicates enabled us a better approximation of cis landscape within species and also provides a resource for future exploration for better models.

Corrigendum to “Marginal Effects in Interaction Models: Determining and Controlling the False Positive Rate”

2020 ◽  
pp. 001041402097021
Author(s):  
Justin Esarey ◽  
Jane L. Sumner
Keyword(s):  
False Positive ◽  
False Positive Rate ◽  
Interaction Models ◽  
Multiple Hypotheses ◽  
Marginal Effects ◽  
Multiple Hypothesis ◽  
Positive Rate

After publication of Esarey and Sumner, we recognized substantively significant errors in the paper. We correct those errors here. We discovered these errors as part of work on another, unrelated paper to which one co-author hoped to apply similar methods. The Brambor et al. procedure is overconfident when separately testing multiple hypotheses, as correctly stated in our paper. However, when conjointly testing multiple hypotheses, the Brambor et al. procedure is appropriate. The procedure we suggest on pp. 1161–1163 to correct for overconfidence in the case of separate testing of multiple hypothesis (based on Benjamini and Hochberg) is (to our knowledge) correct, but subject to several limitations unstated in the paper. This corrigendum lays out all errors and limitations and adds a more robust procedure to our interactionTest software.

Improving diagnosis of acute appendicitis with atypical findings by Tc-99m HMPAO leukocyte scan

2002 ◽  
Vol 41 (01) ◽  
pp. 37-41 ◽  
Author(s):  
S. Shung-Shung ◽  
S. Yu-Chien ◽  
Y. Mei-Due ◽  
W. Hwei-Chung ◽  
A. Kao
Keyword(s):  
Acute Appendicitis ◽  
False Positive ◽  
False Positive Rate ◽  
Accurate Method ◽  
Clinical Findings ◽  
Pathological Findings ◽  
Lower Quadrant ◽  
Predictive Values ◽  
Positive Rate

Summary Aim: Even with careful observation, the overall false-positive rate of laparotomy remains 10-15% when acute appendicitis was suspected. Therefore, the clinical efficacy of Tc-99m HMPAO labeled leukocyte (TC-WBC) scan for the diagnosis of acute appendicitis in patients presenting with atypical clinical findings is assessed. Patients and Methods: Eighty patients presenting with acute abdominal pain and possible acute appendicitis but atypical findings were included in this study. After intravenous injection of TC-WBC, serial anterior abdominal/pelvic images at 30, 60, 120 and 240 min with 800k counts were obtained with a gamma camera. Any abnormal localization of radioactivity in the right lower quadrant of the abdomen, equal to or greater than bone marrow activity, was considered as a positive scan. Results: 36 out of 49 patients showing positive TC-WBC scans received appendectomy. They all proved to have positive pathological findings. Five positive TC-WBC were not related to acute appendicitis, because of other pathological lesions. Eight patients were not operated and clinical follow-up after one month revealed no acute abdominal condition. Three of 31 patients with negative TC-WBC scans received appendectomy. They also presented positive pathological findings. The remaining 28 patients did not receive operations and revealed no evidence of appendicitis after at least one month of follow-up. The overall sensitivity, specificity, accuracy, positive and negative predictive values for TC-WBC scan to diagnose acute appendicitis were 92, 78, 86, 82, and 90%, respectively. Conclusion: TC-WBC scan provides a rapid and highly accurate method for the diagnosis of acute appendicitis in patients with equivocal clinical examination. It proved useful in reducing the false-positive rate of laparotomy and shortens the time necessary for clinical observation.

Predicting Fetal Chromosome Anomalies in the First Trimester Using Pregnancy Associated Plasma Protein-A: A Comparison of Statistical Methods

1993 ◽  
Vol 32 (02) ◽  
pp. 175-179 ◽  
Author(s):  
B. Brambati ◽  
T. Chard ◽  
J. G. Grudzinskas ◽  
M. C. M. Macintosh
Keyword(s):  
Logistic Regression ◽  
General Population ◽  
Likelihood Ratio ◽  
False Positive ◽  
False Positive Rate ◽  
Ratio Method ◽  
Detection Rates ◽  
Gaussian Distributions ◽  
Positive Rate ◽  
Likelihood Ratio Method

Abstract:The analysis of the clinical efficiency of a biochemical parameter in the prediction of chromosome anomalies is described, using a database of 475 cases including 30 abnormalities. A comparison was made of two different approaches to the statistical analysis: the use of Gaussian frequency distributions and likelihood ratios, and logistic regression. Both methods computed that for a 5% false-positive rate approximately 60% of anomalies are detected on the basis of maternal age and serum PAPP-A. The logistic regression analysis is appropriate where the outcome variable (chromosome anomaly) is binary and the detection rates refer to the original data only. The likelihood ratio method is used to predict the outcome in the general population. The latter method depends on the data or some transformation of the data fitting a known frequency distribution (Gaussian in this case). The precision of the predicted detection rates is limited by the small sample of abnormals (30 cases). Varying the means and standard deviations (to the limits of their 95% confidence intervals) of the fitted log Gaussian distributions resulted in a detection rate varying between 42% and 79% for a 5% false-positive rate. Thus, although the likelihood ratio method is potentially the better method in determining the usefulness of a test in the general population, larger numbers of abnormal cases are required to stabilise the means and standard deviations of the fitted log Gaussian distributions.

Identification of and Correction for Publication Bias: Comment

2019 ◽  
Author(s):  
Amanda Kvarven ◽  
Eirik Strømland ◽  
Magnus Johannesson
Keyword(s):  
Publication Bias ◽  
False Positive ◽  
Large Scale ◽  
Meta Analysis ◽  
False Positive Rate ◽  
Effect Sizes ◽  
Replication Studies ◽  
Moderate Reduction ◽  
Positive Rate ◽  
Meta Analyses

Andrews & Kasy (2019) propose an approach for adjusting effect sizes in meta-analysis for publication bias. We use the Andrews-Kasy estimator to adjust the result of 15 meta-analyses and compare the adjusted results to 15 large-scale multiple labs replication studies estimating the same effects. The pre-registered replications provide precisely estimated effect sizes, which do not suffer from publication bias. The Andrews-Kasy approach leads to a moderate reduction of the inflated effect sizes in the meta-analyses. However, the approach still overestimates effect sizes by a factor of about two or more and has an estimated false positive rate of between 57% and 100%.

scholarly journals PRATD: A Phased Remote Access Trojan Detection Method with Double-Sided Features

Electronics ◽  
2020 ◽  
Vol 9 (11) ◽  
pp. 1894
Author(s):  
Chun Guo ◽  
Zihua Song ◽  
Yuan Ping ◽  
Guowei Shen ◽  
Yuhei Cui ◽  
...  
Keyword(s):  
False Positive ◽  
Detection Method ◽  
False Positive Rate ◽  
True Positive Rate ◽  
Remote Access ◽  
Detection Methods ◽  
Security Threats ◽  
True Positive ◽  
Trojan Detection ◽  
Positive Rate

Remote Access Trojan (RAT) is one of the most terrible security threats that organizations face today. At present, two major RAT detection methods are host-based and network-based detection methods. To complement one another’s strengths, this article proposes a phased RATs detection method by combining double-side features (PRATD). In PRATD, both host-side and network-side features are combined to build detection models, which is conducive to distinguishing the RATs from benign programs because that the RATs not only generate traffic on the network but also leave traces on the host at run time. Besides, PRATD trains two different detection models for the two runtime states of RATs for improving the True Positive Rate (TPR). The experiments on the network and host records collected from five kinds of benign programs and 20 famous RATs show that PRATD can effectively detect RATs, it can achieve a TPR as high as 93.609% with a False Positive Rate (FPR) as low as 0.407% for the known RATs, a TPR 81.928% and FPR 0.185% for the unknown RATs, which suggests it is a competitive candidate for RAT detection.

Ascertaining an efficient eligibility cut-off for extended Medicare items for eating disorders

2021 ◽  
pp. 103985622110286
Author(s):  
Tracey Wade ◽  
Jamie-Lee Pennesi ◽  
Yuan Zhou
Keyword(s):  
Eating Disorders ◽  
Eating Disorder ◽  
False Positive Rate ◽  
Area Under The Curve ◽  
Rate Sensitivity ◽  
True Positive Rate ◽  
Eating Disorder Examination Questionnaire ◽  
Eating Disorder Examination ◽  
Positive Rate ◽  
The Relationship

Objective: Currently eligibility for expanded Medicare items for eating disorders (excluding anorexia nervosa) require a score ⩾ 3 on the 22-item Eating Disorder Examination-Questionnaire (EDE-Q). We compared these EDE-Q “cases” with continuous scores on a validated 7-item version of the EDE-Q (EDE-Q7) to identify an EDE-Q7 cut-off commensurate to 3 on the EDE-Q. Methods: We utilised EDE-Q scores of female university students ( N = 337) at risk of developing an eating disorder. We used a receiver operating characteristic (ROC) curve to assess the relationship between the true-positive rate (sensitivity) and the false-positive rate (1-specificity) of cases ⩾ 3. Results: The area under the curve showed outstanding discrimination of 0.94 (95% CI: .92–.97). We examined two specific cut-off points on the EDE-Q7, which included 100% and 87% of true cases, respectively. Conclusion: Given the EDE-Q cut-off for Medicare is used in conjunction with other criteria, we suggest using the more permissive EDE-Q7 cut-off (⩾2.5) to replace use of the EDE-Q cut-off (⩾3) in eligibility assessments.

Sign in / Sign up

Export Citation Format

Share Document