scholarly journals Modelling science trustworthiness under publish or perish pressure

2018 ◽  
Vol 5 (1) ◽  
pp. 171511 ◽  
Author(s):  
David Robert Grimes ◽  
Chris T. Bauch ◽  
John P. A. Ioannidis
Keyword(s):  
Phenomenological Model ◽  
False Positive ◽  
Null Hypothesis ◽  
False Positive Rate ◽  
Scientific Publication ◽  
Negative Consequences ◽  
Publish Or Perish ◽  
Positive Rate ◽  
Positive Results ◽  
Scientific Endeavour

Scientific publication is immensely important to the scientific endeavour. There is, however, concern that rewarding scientists chiefly on publication creates a perverse incentive, allowing careless and fraudulent conduct to thrive, compounded by the predisposition of top-tier journals towards novel, positive findings rather than investigations confirming null hypothesis. This potentially compounds a reproducibility crisis in several fields, and risks undermining science and public trust in scientific findings. To date, there has been comparatively little modelling on factors that influence science trustworthiness, despite the importance of quantifying the problem. We present a simple phenomenological model with cohorts of diligent, careless and unethical scientists, with funding allocated by published outputs. This analysis suggests that trustworthiness of published science in a given field is influenced by false positive rate, and pressures for positive results. We find decreasing available funding has negative consequences for resulting trustworthiness, and examine strategies to combat propagation of irreproducible science.

scholarly journals Modeling science trustworthiness under publish or perish pressure

2017 ◽  
Author(s):  
David Robert Grimes ◽  
Chris T. Bauch ◽  
John P.A. Ioannidis
Keyword(s):  
Intrinsic Value ◽  
False Positive Rate ◽  
Scientific Publication ◽  
Negative Consequences ◽  
Publish Or Perish ◽  
Negative Results ◽  
Positive Rate ◽  
Scientific Endeavor ◽  
Positive Results ◽  
Accurate Reporting

AbstractThe scientific endeavor pivots on the accurate reporting of experimental and theoretical findings, and consequently scientific publication is immensely important. As the number of active scientists continues to increase, there is concern that rewarding scientists chiefly on publication creates a perverse incentive where careless and fraudulent research can thrive. This is compounded by the predisposition of top-tier journals towards novel or positive findings rather than negative results or investigations that merely confirm a null hypothesis, despite their intrinsic value, potentially compounding a reproducibility crisis in several fields. This is a serious problem for both science and public trust in scientific findings. To date, there has been comparatively little mathematical modeling on the factors that influence science trustworthiness, despite the importance of quantifying the problem. In this work, we present a simple phenomenological model with cohorts of diligent, careless and unethical scientists with funding allocated based on published outputs. The results of this analysis suggest that trustworthiness of published science in a given field is strongly influenced by the false positive rate and the pressures from journals for positive results, and that decreasing available funding has negative consequences for the resulting trustworthiness. We also examine strategies to combat propagation of irreproducible science, including increasing fraud detection and awarding diligence, discussing the implications of these findings.

Impact of Biases in the False-Positive Rate on Null Hypothesis Testing

Epilepsy ◽  
2011 ◽  
pp. 241-248
Author(s):  
Ralph Andrzejak ◽  
Daniel Chicharro ◽  
Florian Mormann
Keyword(s):  
Hypothesis Testing ◽  
False Positive ◽  
Null Hypothesis ◽  
False Positive Rate ◽  
Null Hypothesis Testing ◽  
Positive Rate

Properly collected plasma metanephrines excludes PPGL after false positive screening tests

2021 ◽  
Author(s):  
Gregory A Kline ◽  
Jessica Boyd ◽  
Brenda Polzin ◽  
Adrian Harvey ◽  
Janice L Pasieka ◽  
...  
Keyword(s):  
False Positive ◽  
Adrenal Mass ◽  
Chart Review ◽  
False Positive Rate ◽  
Total Positivity ◽  
Screening Tests ◽  
Positive Rate ◽  
Plasma Metanephrines ◽  
Positive Results ◽  
Urine Metanephrines

Abstract Context False positive results are common for pheochromocytoma/paraganglioma(PPGL) real-world screening. Objective Determine the correlation between screening urine and seated plasma metanephrines in outpatients where PPGL was absent, compared to meticulously prepared and supine-collected plasma metanephrines with age-adjusted references. Design Retrospective cohort study Setting Databases from a single-provider provincial laboratory(2012-2018), a validated PPGL registry and a manual chart review from a specialized endocrine testing unit. Patients PPGL registry data excluded known PPGL cases from the laboratory database. Outpatients having both urine and plasma metanephrines <90 days apart. Methods The correlation between urine and seated plasma measures along with the total positivity rate. All cases of plasma metanephrines drawn in the endocrine unit were reviewed for test indication and test positivity rate. Results There were 810 non-PPGL pairs of urine and plasma metanephrines in the laboratory database; 46.1% of urine metanephrines were reported high. Of seated outpatient plasma metanephrines drawn a median of 5.9 days later, 19.2% were also high (r=0.33 and 0.50 for normetanephrine and metanephrine, respectively). In contrast, the meticulously prepared and supine collected patients(n=139, 51% prior high urine metanephrines) had <3% rate of abnormal high results in patients without known PPGL/adrenal mass. Conclusions There was a poor-to-moderate correlation between urine and seated plasma metanephrines. Up to 20% of those with high urine measures also had high seated plasma metanephrines in the absence of PPGL. Properly prepared and collected supine plasma metanephrines had a false positive rate of <3% in the absence of known PPGL/adrenal mass.

Validation of Receptor-Binding Assays To Detect Antibiotics in Goat's Milk

2014 ◽  
Vol 77 (2) ◽  
pp. 308-313 ◽  
Author(s):  
M. C. BELTRÁN ◽  
M. BORRÀS ◽  
O. NAGEL ◽  
R. L. ALTHAUS ◽  
M. P. MOLINA
Keyword(s):  
Receptor Binding ◽  
False Positive ◽  
False Positive Rate ◽  
Quality Parameters ◽  
Lactation Period ◽  
Binding Assays ◽  
Goat’S Milk ◽  
Positive Rate ◽  
Goat's Milk ◽  
Positive Results

The suitability of different receptor-binding assays to detect antibiotics in raw goat's milk was investigated. Detection capability of most β-lactams and tetracyclines assessed applying the Betastar Combo, the SNAP Betalactam, the SNAP Tetracycline, and the Twinsensor tests was at or below maximum residue limits established by European legislation. Regarding test specificity, cross-reactions with antibiotics other than β-lactams and tetracyclines were not found, and no false-positive results were obtained for the Betastar Combo and the SNAP tests when bulk samples of goat's milk were analyzed. For the Twinsensor test, the false-positive rate was 1%. The performance of the Betastar Combo and the SNAP tests was practically unaffected by the milk quality parameters using individual samples of goat's milk collected at points throughout the entire lactation period (false-positive rate, ≤5%). However, a larger number of positive results were obtained by the Twinsensor test in this type of milk sample (>10%), especially in the last weeks of lactation. Interferences related to the use of the preservative azidiol were not observed in any case. Neither were any significant differences found in relation to the interpretation method (visual versus instrumental) applied. In general, the response of the Betastar Combo, SNAP, and Twinsensor tests was optimal for the analysis of bulk caprine milk; thus, they may be used to monitor milk for the presence of β-lactam and tetracycline residues in quality control programs.

scholarly journals Use of a Disposable Water Filter for Prevention of False-Positive Results due to Nontuberculosis Mycobacteria in a Clinical Laboratory Performing Routine Acid-Fast Staining for Tuberculosis

2007 ◽  
Vol 73 (19) ◽  
pp. 6296-6298 ◽  
Author(s):  
Hui-Zin Tu ◽  
Chiao-Shan Chen ◽  
Tsi-Shu Huang ◽  
Wen-Kuei Huang ◽  
Yao-shen Chen ◽  
...  
Keyword(s):  
False Positive ◽  
Clinical Laboratory ◽  
Tap Water ◽  
False Positive Rate ◽  
Acid Fast Bacillus ◽  
Water Filter ◽  
Point Of Use ◽  
Positive Rate ◽  
Positive Results ◽  
Laboratory Water

ABSTRACT A point-of-use 0.2-μm filter was evaluated for elimination of nontuberculosis mycobacteria in laboratory water to reduce false-positive acid-fast bacillus staining results. Use of the point-of-use filter can significantly reduce the false-positive rate to 1.2% compared to samples treated with tap water (10.7%) and deionized water (8.7%).

scholarly journals Commercial Laboratory IgM Testing forToxoplasma gondiiin Pregnancy: A 20-Year Experience

2005 ◽  
Vol 13 (3) ◽  
pp. 151-153 ◽  
Author(s):  
David J. Garry ◽  
Andrew Elimian ◽  
Vandy Wiencek ◽  
David A. Baker
Keyword(s):  
Predictive Value ◽  
False Positive ◽  
False Positive Rate ◽  
Reference Laboratory ◽  
Antibody Testing ◽  
Commercial Laboratory ◽  
Laboratory Results ◽  
Positive Rate ◽  
Acute Toxoplasmosis ◽  
Positive Results

Objective.This study was performed to review the clinical utility of commercial laboratoryToxoplasmosis-specific IgM testing during pregnancy and outcomes of the gestation at our institution.Methods.A retrospective review of all women referred for suspected acuteToxoplasma gondiiinfection during pregnancy from 1984 through 2004 was performed. Women were diagnosed with suspected acute toxoplasmosis based on commercial laboratory serologic antibody testing. All women had blood sent to a recognized reference laboratory for antibody testing within 2 weeks of the commercial laboratory results. The study protocol was approved by the Institutional Review Board. Chi-square analysis were used with a significance ofP< .05.Results.A total of 130 women were evaluated during the study period with 116 IgM positive results from the commercial laboratories. The commercial laboratory antibodies were as follows: IgM positive with IgG negative (n= 20), IgM positive with IgG positive (n= 96), and IgM negative with IgG positive (n= 14). There was a significant reduction in the IgM positive results when comparing commercial laboratory (n= 116) with the reference laboratory results (n= 28;p< .001). Acute toxoplasmosis infection was diagnosed in 7 (5%) of the women. All cases of acute toxoplasmosis infection had a positive commercial laboratory IgM result. The false positive rate for the commercial laboratory IgM was 88.6% and the diagnostic indices were sensitivity 100%, specificity 11.4%, positive predictive value 6% and negative predictive value 100%.Conclusion.Commercial laboratoryToxoplasmosis-specific IgM is associated with a high false positive rate. The commercial and reference laboratory IgM results identified all cases of acute toxoplasmosis infection. Commercial laboratories reflexively obtaining reference laboratory confirmation of positive results could reduce costs associated with testing, referrals, retesting, and invasive procedures.

scholarly journals False Positive Cardiotoxicity Events in Cancer-Related Clinical Trials: Risks Related to Imperfect Noninvasive Parameters

Chemotherapy ◽  
2018 ◽  
Vol 63 (6) ◽  
pp. 324-329 ◽  
Author(s):  
Michael S. Ewer ◽  
Jay Herson
Keyword(s):  
False Positive ◽  
False Positive Rate ◽  
Left Ventricular ◽  
False Positives ◽  
Left Ventricular Ejection ◽  
True Positive ◽  
Ventricular Ejection Fraction ◽  
Positive Rate ◽  
Estimation Of Variance ◽  
Positive Results

Purpose: Cardiac ultrasound provides important structural and functional information that makes identification of cardiac abnormalities possible. Left ventricular ejection fraction (LVEF) provides the most commonly used parameter for recognition of treatment-related cardiac dysfunction. Random reading variance and physiologic factors influence LVEF and make the reported value imperfect. We attempt to quantitate the likelihood of false positive events by computer simulation. Methods: We simulated four visits on hypothetical trials. We assumed a baseline LVEF of 55% and normal distribution with regard to reading error and physiologic variation. 1,000 trials of sample size 1,500 were simulated. In a separate simulation, 1,000 patients entered with LVEFs of 45, 43, and 41% to estimate true positive incidence. Results: At each examination, less than 1.0% of false positives were noted. The cumulative false positive rate over four visits was 3.60%. True cardiotoxicity identification is satisfactory only when LVEF declines substantially. Conclusion: A 3.60% false positive rate in trials where the expected level of toxicity is low suggests that false positives are troubling and may exceed true positive results. Strategies to reduce the number of false positive results include making confirmatory studies mandatory. Evaluating increases along with decreases obtains some estimation of variance.

scholarly journals A method to alleviate false-positive results of the Elecsys HIV combi PT assay

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Xiaolan Lu ◽  
Minghong Zhang ◽  
Wen Liu ◽  
Nan Sheng ◽  
Qin Du ◽  
...  
Keyword(s):  
Hiv Infection ◽  
Retrospective Analysis ◽  
False Positive ◽  
False Positive Rate ◽  
Hiv Screening ◽  
Screening Assay ◽  
Positive Serum ◽  
Positive Rate ◽  
The Difference ◽  
Positive Results

AbstractTo explore the effects of urea dissociation on reducing false-positive results of  the Elecsys HIV combi PT assay. A retrospective analysis was used to evaluate the false-positive rate of the Elecsys HIV combi PT assay. Six false-positive sera, six positive sera and six sera from patients with early HIV infection were collected. Dissociation was performed using 1 mol/L, 2 mol/L, 4 mol/L, 6 mol/L, or 8 mol/L urea, and HIV screening assay were then detected to select the appropriate concentration of urea dissociation. Next, 55 false-positive sera and 15 sera from early HIV infection were used to verify the best concentration of urea to achieve dissociation. Retrospective analysis showed that the COI of the Elecsys HIV combi PT assay in false-positive sera ranged from 1.0 to 200.0, and approximately 97.01%(227/234) of false-positive sera were in the range of 1.0–15.0. The avidity index (AI) in positive and false-positive sera decreased as the urea dissociation concentration increased. When the dissociation concentration was 6 mol/L, the AI of false-positive serum was between 0.0234 and 0.2567, and the AI of early HIV infection sera was between 0.4325 and 0.5017. The difference in AI between false-positive and positive samples was significant. When negativity was defined as an AI of less than 0.3970, the sensitivity and specificity were 100.0% and 100.0%, respectively. Urea-mediated dissociation could significantly reduce the false-positive rate of the Elecsys HIV combi PT assay with a low COI. Our findings provided a reference for distinguishing positive and false-positive of the Elecsys HIV combi PT assay.

Incidence of HCV and HIV NAT Positive in Cord Blood Donors.

Blood ◽  
2005 ◽  
Vol 106 (11) ◽  
pp. 1900-1900
Author(s):  
Jennifer Rhamy ◽  
Mark Beltz ◽  
Karla John ◽  
David Feria ◽  
Dan Waxman ◽  
...  
Keyword(s):  
Cord Blood ◽  
False Positive ◽  
Blood Donors ◽  
False Positive Rate ◽  
True Positive ◽  
Disease Markers ◽  
Positive Antibody ◽  
Positive Rate ◽  
Two Populations ◽  
Positive Results

Abstract Introduction: Donors of cord blood (CBD) are screened similarly to whole blood donors (WBD). It has been reported that CBD have a higher incidence of both true positive and false positive disease markers tested by EIA[1]. This abstract compares the rate of positive results for viral RNA (NAT) in CBD vs. WBD in a large cohort of donors. Methods: NAT was performed using the Chiron Procleix MPX or discriminatory assays. EIA testing was performed using Abbott bead methodology following manufacturer’s instructions. Confirmatory results for EIA were performed with Chiron RIBA. Results from 12/17/04 to 6/30/05 were collated. Results EIA HIV EIA HCV NAT HIV/HCV HIV EIA POS(%) HCV EIA POS (%) HIV NAT POS HCV NAT POS WBD 74299 74299 74299 75(0.10) 107(0.14) 1 (0) 31(0.04) CBD 11996 12001 12007 50(0.42) 46(0.38) 0 13 (0.11) A higher percentage of CBD were positive for HIV and HCV EIA as previously reported1. HIV NAT positive donors were 0.00% for both populations. The ratio of HCV EIA positive to HCV NAT positive donors was 28.97% for WBD and 28.26% in CBD. The ratio of CBD HCV EIA positives to WBD HCV EIA positives is 2.71 and the ratio of WBD HCV NAT positives to WBD HCV NAT positives is 2.75. The ratio of HIV CBD EIA positives to HIV WBD EIA positives is 4.10, which is significantly higher (p=<0.01). Conclusions Both CBD and WBD have a low rate (0%) of HIV NAT positive donors, but CBD have a 4x higher rate of HIV EIA. This suggests a 4x higher donor loss to false positive antibody results. The CBD donors have a 2.7x higher rate for both EIA and NAT positive results, showing a higher rate of viremic donors in CBD. The similar ratios for EIA positive to NAT positive donors in the two populations suggest that the false positive rate is not elevated in CBD versus WBD. The higher incidence of HCV infection in CBD could be geographic or ethnic in origin. More study is needed to determine the reasons and further predict donor loss. The HIV EIA results are not predictive of actual infection in this population. A more aggressive logarithm for reentry than used for WBD may be merited.

Test Performance and Test-Retest Reliability of the Vestibular/Ocular Motor Screening and King-Devick Test in Adolescent Athletes During a Competitive Sport Season

2018 ◽  
Vol 46 (8) ◽  
pp. 2004-2010 ◽  
Author(s):  
Phillip R. Worts ◽  
Philip Schatz ◽  
Scott O. Burkhart
Keyword(s):  
High School ◽  
High School Student ◽  
False Positive ◽  
False Positive Rate ◽  
Ocular Motor ◽  
Retest Reliability ◽  
Positive Rate ◽  
Test Retest Reliability ◽  
Positive Results ◽  
Level Of Agreement

Background: The Vestibular/Ocular Motor Screening (VOMS) and King-Devick (K-D) test are tools designed to assess ocular or vestibular function after a sport-related concussion. Purpose: To determine the test-retest reliability and rate of false-positive results of the VOMS and K-D test in a healthy athlete sample. Study Design: Cohort study (diagnosis); Level of evidence, 2. Methods: Forty-five healthy high school student-athletes (mean age, 16.11 ± 1.43 years) completed self-reported demographics and medical history and were administered the VOMS and K-D test during rest on day 1 (baseline). The VOMS and K-D test were administered again once during rest (prepractice) and once within 5 minutes of removal from sport practice on day 2 (removal). The Borg rating of perceived exertion scale was administered at removal. Intraclass correlation coefficients were used to determine test-retest reliability on the K-D test and the average near point of convergence (NPC) distance on the VOMS. Level of agreement was used to examine VOMS symptom provocation over the 3 administration times. Multivariate base rates were used to determine the rate of false-positive results when simultaneously considering multiple clinical cutoffs. Results: Test-retest reliability of total time on the K-D test (0.91 [95% CI, 0.86-0.95]) and NPC distance (0.91 [95% CI, 0.85-0.95]) was high across the 3 administration times. Level of agreement ranged from 48.9% to 88.9% across all 3 times for the VOMS items. Using established clinical cutoffs, false-positive results occurred in 2% of the sample using the VOMS at removal and 36% using the K-D test. Conclusion: The VOMS displayed a false-positive rate of 2% in this high school student-athlete cohort. The K-D test’s false-positive rate was 36% while maintaining a high level of test-retest reliability (0.91). Results from this study support future investigation of VOMS administration in an acutely injured high school athletic sample. Going forward, the VOMS may be more stable than other neurological and symptom report screening measures and less vulnerable to false-positive results than the K-D test.

Sign in / Sign up

Export Citation Format

Share Document