scholarly journals The natural selection of bad science

2016 ◽  
Vol 3 (9) ◽  
pp. 160384 ◽  
Author(s):  
Paul E. Smaldino ◽  
Richard McElreath
Keyword(s):  
Natural Selection ◽  
Statistical Power ◽  
Meta Analysis ◽  
Computational Modelling ◽  
False Discovery Rates ◽  
Culture Of Science ◽  
Quality Of Research ◽  
Selection Of ◽  
High Output

Poor research design and data analysis encourage false-positive findings. Such poor methods persist despite perennial calls for improvement, suggesting that they result from something more than just misunderstanding. The persistence of poor methods results partly from incentives that favour them, leading to the natural selection of bad science. This dynamic requires no conscious strategizing—no deliberate cheating nor loafing—by scientists, only that publication is a principal factor for career advancement. Some normative methods of analysis have almost certainly been selected to further publication instead of discovery. In order to improve the culture of science, a shift must be made away from correcting misunderstandings and towards rewarding understanding. We support this argument with empirical evidence and computational modelling. We first present a 60-year meta-analysis of statistical power in the behavioural sciences and show that power has not improved despite repeated demonstrations of the necessity of increasing power. To demonstrate the logical consequences of structural incentives, we then present a dynamic model of scientific communities in which competing laboratories investigate novel or previously published hypotheses using culturally transmitted research methods. As in the real world, successful labs produce more ‘progeny,’ such that their methods are more often copied and their students are more likely to start labs of their own. Selection for high output leads to poorer methods and increasingly high false discovery rates. We additionally show that replication slows but does not stop the process of methodological deterioration. Improving the quality of research requires change at the institutional level.

scholarly journals Open science and modified funding lotteries can impede the natural selection of bad science

2019 ◽  
Vol 6 (7) ◽  
pp. 190194 ◽  
Author(s):  
Paul E. Smaldino ◽  
Matthew A. Turner ◽  
Pablo A. Contreras Kallens
Keyword(s):  
Natural Selection ◽  
Peer Review ◽  
Computational Modelling ◽  
Open Science ◽  
Strategic Behaviour ◽  
Negative Results ◽  
Recent Developments ◽  
False Discoveries ◽  
Selection Of

Assessing scientists using exploitable metrics can lead to the degradation of research methods even without any strategic behaviour on the part of individuals, via ‘the natural selection of bad science.’ Institutional incentives to maximize metrics like publication quantity and impact drive this dynamic. Removing these incentives is necessary, but institutional change is slow. However, recent developments suggest possible solutions with more rapid onsets. These include what we call open science improvements , which can reduce publication bias and improve the efficacy of peer review. In addition, there have been increasing calls for funders to move away from prestige- or innovation-based approaches in favour of lotteries. We investigated whether such changes are likely to improve the reproducibility of science even in the presence of persistent incentives for publication quantity through computational modelling. We found that modified lotteries, which allocate funding randomly among proposals that pass a threshold for methodological rigour, effectively reduce the rate of false discoveries, particularly when paired with open science improvements that increase the publication of negative results and improve the quality of peer review. In the absence of funding that targets rigour, open science improvements can still reduce false discoveries in the published literature but are less likely to improve the overall culture of research practices that underlie those publications.

scholarly journals Terminomics Methodologies and the Completeness of Reductive Dimethylation: A Meta-Analysis of Publicly Available Datasets

Proteomes ◽  
2019 ◽  
Vol 7 (2) ◽  
pp. 11 ◽  
Author(s):  
Mariella Hurtado Silva ◽  
Iain Berry ◽  
Natalie Strange ◽  
Steven Djordjevic ◽  
Matthew Padula
Keyword(s):  
Meta Analysis ◽  
Unexpected Finding ◽  
Common Occurrence ◽  
Available N ◽  
Historical Literature ◽  
Previous Decade ◽  
Amine Groups ◽  
Selection Of

Methods for analyzing the terminal sequences of proteins have been refined over the previous decade; however, few studies have evaluated the quality of the data that have been produced from those methodologies. While performing global N-terminal labelling on bacteria, we observed that the labelling was not complete and investigated whether this was a common occurrence. We assessed the completeness of labelling in a selection of existing, publicly available N-terminomics datasets and empirically determined that amine-based labelling chemistry does not achieve complete labelling and potentially has issues with labelling amine groups at sequence-specific residues. This finding led us to conduct a thorough review of the historical literature that showed that this is not an unexpected finding, with numerous publications reporting incomplete labelling. These findings have implications for the quantitation of N-terminal peptides and the biological interpretations of these data.

CURRENT BEST PRACTICES AND BIOBANKING RECOMMENDATIONS

2019 ◽  
Vol 64 (12) ◽  
pp. 769-776 ◽  
Author(s):  
Y. V. Doludin ◽  
A. L. Borisova ◽  
M. S. Pokrovskaya ◽  
O. V. Stefanyuk ◽  
O. V. Sivakova ◽  
...  
Keyword(s):  
Best Practices ◽  
Preanalytical Phase ◽  
Wide Range ◽  
Research Problems ◽  
Processing And Storage ◽  
And Storage ◽  
Quality Of Research ◽  
Selection Of

The biobank is a structure established with the goal of long-term responsible storage of biological samples and the associated data for their further use in scientific and clinical research. The objectives of biobanking are the creation of unified recommendations on: the planning of premises and the selection of equipment for storage; development of management methods and staff training; standardization of methods for the collection, shipping, processing and storage of biomaterial of various origins, as well as methods for quality control and validation of the applied methods; creation and use of databases of information accompanying biospecimens. The lack of common standards for conducting the preanalytical phase has been the cause of low accuracy and poor reproducibility of research results. To date, a large number of guidelines and best practices have been published that provide an answer to a wide range of problems in organizing the biobanking process. The article provides an overview of the most famous biobanking guidelines that can be used to solve various research problems. Biobanking in Russia is actively developing. Since 1996 there is a work on the legislative regulation of biobanking activities, as a result of which a number of regulatory documents have been issued. An important stage in the development of biobanking in Russia was the establishment of the “National Association of Biobanks and Biobanking Specialists” (NASBio) in 2018, which included representatives of medical and research institutions, commercial firms, and qualified specialists in the field of biobanking. One of the key tasks of NASBio is the adaptation and implementation of the best biobanking practices in Russian research institutes and centers. The use of modern guidelines and best practices on biobanking will lead to an increase in the quality of research and publications.

Meta-analysis of Comparative Therapy Outcome Studies: a Reply to Wilson

1982 ◽  
Vol 10 (4) ◽  
pp. 307-310 ◽  
Author(s):  
David A. Shapiro ◽  
Diana Shapiro
Keyword(s):  
Meta Analysis ◽  
Outcome Studies ◽  
Therapy Outcome ◽  
Meta Analyses ◽  
Selection Of

Wilson's recent critique of the authors' appraisal of meta-analysis appears to misunderstand them as claiming more for meta-analysis than they intended. The present paper seeks to clarify consequent confusions concerning selection of studies, the quality of the literature reviewed, the classification of therapies, and the non-identical results of different meta-analyses. It is acknowledged that no single meta-analysis is definitive.

scholarly journals Predictors of efficacy in depression prevention programmes

2003 ◽  
Vol 183 (5) ◽  
pp. 384-397 ◽  
Author(s):  
Eva Jané-Llopis ◽  
Clemens Hosman ◽  
Rachel Jenkins ◽  
Peter Anderson
Keyword(s):  
Depressive Symptoms ◽  
Research Design ◽  
Psychiatric Symptoms ◽  
Meta Analysis ◽  
Age Groups ◽  
High Quality ◽  
Prevention Programmes ◽  
Quality Of Research ◽  
Different Levels

BackgroundWorldwide, 340 million people are affected by depression, with high comorbid, social and economic costs.AimsTo identify potential predictors of effect in prevention programmes.MethodA meta-analysis was made of 69 programmes to reduce depression or depressive symptoms.ResultsThe weighted mean effect size of 0.22 was effective for different age groups and different levels of risk, and in reducing risk factors and depressive or psychiatric symptoms. Programmes with larger effect sizes were multi-component, included competence techniques, had more than eight sessions, had sessions 60–90 min long, had a high quality of research design and were delivered by a health care provider in targeted programmes. Older people benefited from social support, whereas behavioural methods were detrimental.ConclusionsAn 11% improvement in depressive symptoms can be achieved through prevention programmes. Single trial evaluations should ensure high quality of the research design and detailed reporting of results and potential predictors.

Do Teacher Attitudes Impact TeachingThrough Information Technology

2017 ◽  
Vol 4 (24) ◽  
Author(s):  
Suman Kumari Katoch
Keyword(s):  
Information Technology ◽  
Study Data ◽  
Survey Method ◽  
School Teachers ◽  
Second Stage ◽  
Good Sample ◽  
Two Stages ◽  
Quality Of Research ◽  
Selection Of

Teachers’ styles, and mainly their attitudes, are strong context outcomes, rooted in experience and do not become automatic routine conducts, in the sense that they are developed via very slow interactions and become well established constructs for each individual only after some time. In that sense attitudes can be modified only by each individual, when he/she becomes aware, via elements and evidence, that new postures would be better to deal with the world around. In the present study data regarding the attitude of teachers towards information technology was gathered with the help of survey method. All the school teachers of districts Bilaspur and Hamirpur of Himachal Pradesh constituted the population of the study. In order to ensure high quality of research, selection of a good sample is must. For this purpose sampling was done at two stages. At the first stage 14 schools were selected and in the second stage 150 teachers were selected randomly from the each selected school. Keeping in view the nature of the present study the investigator used the standardized tool, “Attitude towards Information Technology Scales. The reliability of the tool is 0.88. To find out the significance of difference between the various groups ‘t’-test was applied. The findings of the study revealed that gender-wise, locality-wise, school teachers do not differed significantlyin their attitude towards information technology.

scholarly journals Efficacy and safety of triazoles versus echinocandins in the treatment of invasive aspergillosis: A meta-analysis

2020 ◽  
Vol 77 (9) ◽  
pp. 974-985
Author(s):  
Sanja Uzelac ◽  
Radica Zivkovic-Zaric ◽  
Milan Radovanovic ◽  
Goran Rankovic ◽  
Slobodan Jankovic
Keyword(s):  
Clinical Trials ◽  
Invasive Aspergillosis ◽  
Therapeutic Option ◽  
Meta Analysis ◽  
Treatment Success ◽  
Poor Quality ◽  
Efficacy And Safety ◽  
Set Inclusion ◽  
Selection Of

Backgroun/Aim. Although majority of guidelines recommend triazoles (voriconazole, posaconazole, itraconazole and isavuconazole) as first-line therapeutic option for treatment of invasive aspergillosis, echinocandins (caspofungin, micafungin and anidulafungin) are also used for this purpose. However, head-to-head comparison of triazoles and echinocandins for invasive aspergillosis was rarely target of clinical trials. The aim of this meta-analysis was to compare efficacy and safety of triazoles and echinocandins when used for treatment of patients with invasive aspergillosis. Methods. This meta-analysis was based on systematic search of literature and selection of high-quality evidence according to pre-set inclusion and exclusion criteria. The literature search was made for comparison of treatment with any of triazoles (isavuconazole, itraconazole, posaconazole or voriconazole) versus any of echinocandins (caspofungin, anidulafungin or micafungin). The effects of triazoles (itraconazole, posaconazole or voriconazole) and echinocandins (caspofungin, anidulafungin or micafungin) were summarized using RevMan 5.3.5 software, and heterogeneity assessed by the Cochrane Q test and I? values. Several types of bias were assessed, and publication bias was shown by the funnel plot and Egger?s regression. Results. Two clinical trials and three cohort studies were included in this meta-analysis. Mortality in patients with invasive aspergillosis who were treated with triazoles was significantly lower than in patients treated with echinocandins [odds ratio 0.29 (0.13, 0.67)], and rate of favorable response (overall treatment success) 12 weeks after the therapy onset was higher in patients treated with triazoles [3.05 (1.52, 6.13)]. On the other hand, incidence of adverse events was higher with triazoles than with echinocandins in patients treated for invasive aspergillosis [3.75 (0.89, 15.76)], although this difference was not statistically significant. Conclusion.Triazoles (voriconazole in the first place) could be considered as more effective and somewhat less safe therapeutic option than echinocandins for invasive aspergillosis: However, due to poor quality of studies included in this meta-analysis, definite conclusion should await results of additional, well designed clinical trials.

scholarly journals Antibiotic heteroresistance in Mycobacterium tuberculosis isolates: a systematic review and meta-analysis

2021 ◽  
Vol 20 (1) ◽  
Author(s):  
Mao Ye ◽  
Wen Yuan ◽  
Leila Molaeipour ◽  
Khalil Azizian ◽  
Alireza Ahmadi ◽  
...  
Keyword(s):  
Mycobacterium Tuberculosis ◽  
Subgroup Analysis ◽  
Detection Method ◽  
Meta Analysis ◽  
Resistance Development ◽  
Pooled Prevalence ◽  
Important Trend ◽  
Reliable Methods ◽  
Selection Of

Abstract Background Mycobacterium tuberculosis (MTB) is responsible for tuberculosis; that continues to be a public health threat across the globe. Furthermore, increasing heteroresistance (HR)-the presence of resistant and susceptible isolates among MTB strains- has been reported from around the world. This phenomenon can lead to full resistance development and treatment failure. Methods We systematically searched the relevant studies in PubMed, Scopus, and Embase (Until October 21, 2020). The study outcomes revealed the weighted pooled prevalence of antibiotic HR in MTB isolates with subgroup analysis by year, quality of study, and heteroresistance detection method. Results A total of 38 studies which had investigated MTB isolates were included in the meta-analysis. Geographically, the highest number of studies were reported from Asia (n  =  24), followed by Africa (n  =  5). Nineteen studies reported HR to isoniazid, with a weighted pooled prevalence of 5% (95% CI 0–12) among 11,761 MTB isolates. Also, there is no important trend for the subgroup analysis by the study period (2001–2014 vs 2015–2017 vs 2018–2020). HR to rifampin was reported in 17 studies, with a weighted pooled prevalence of 7% (95% CI 2–14) among 3782 MTB isolates. HR to fluoroquinolone and ethambutol were reported in 12 and 4 studies, respectively, with weighted pooled prevalence of 10% and 1% among 2153 and 1509 MTB isolates, correspondingly. Conclusion Based on our analysis, HR in MTB isolates with different frequency rate is present worldwide. Thus, the selection of appropriate and reliable methods for HR detection is crucial for TB eradication.

scholarly journals Promoting Open Science to Increase the Trustworthiness of Evidence in Special Education

2018 ◽  
Vol 85 (1) ◽  
pp. 104-118 ◽  
Author(s):  
Bryan G. Cook ◽  
John Wills Lloyd ◽  
David Mellor ◽  
Brian A. Nosek ◽  
William J. Therrien
Keyword(s):  
Special Education ◽  
Scientific Evidence ◽  
Open Science ◽  
Cultural Norms ◽  
Policy And Practice ◽  
Individuals With Disabilities ◽  
Science Practices ◽  
Quality Of Research ◽  
Selection Of

Scientific evidence should guide the selection of practice for individuals with disabilities. Scientific evidence, however, must be trustworthy to move special education toward greater empirical certainty and more effective policies and practices. Transparency, openness, and reproducibility increase the trustworthiness of evidence. We propose that researchers in special education adopt emerging open-science reforms, such as preprints, data and materials sharing, preregistration of studies and analysis plans, and Registered Reports. Adoption of these practices will require shifts in cultural norms, guidelines, and incentives. We discuss how adopting open-science practices can advance the quality of research and, consequently, policy and practice in special education.

Sign in / Sign up

Export Citation Format

Share Document