scholarly journals Open science and modified funding lotteries can impede the natural selection of bad science

2019 ◽  
Vol 6 (7) ◽  
pp. 190194 ◽  
Author(s):  
Paul E. Smaldino ◽  
Matthew A. Turner ◽  
Pablo A. Contreras Kallens
Keyword(s):  
Natural Selection ◽  
Peer Review ◽  
Computational Modelling ◽  
Open Science ◽  
Strategic Behaviour ◽  
Negative Results ◽  
Recent Developments ◽  
False Discoveries ◽  
Selection Of

Assessing scientists using exploitable metrics can lead to the degradation of research methods even without any strategic behaviour on the part of individuals, via ‘the natural selection of bad science.’ Institutional incentives to maximize metrics like publication quantity and impact drive this dynamic. Removing these incentives is necessary, but institutional change is slow. However, recent developments suggest possible solutions with more rapid onsets. These include what we call open science improvements , which can reduce publication bias and improve the efficacy of peer review. In addition, there have been increasing calls for funders to move away from prestige- or innovation-based approaches in favour of lotteries. We investigated whether such changes are likely to improve the reproducibility of science even in the presence of persistent incentives for publication quantity through computational modelling. We found that modified lotteries, which allocate funding randomly among proposals that pass a threshold for methodological rigour, effectively reduce the rate of false discoveries, particularly when paired with open science improvements that increase the publication of negative results and improve the quality of peer review. In the absence of funding that targets rigour, open science improvements can still reduce false discoveries in the published literature but are less likely to improve the overall culture of research practices that underlie those publications.

scholarly journals Open science and modified funding lotteries can impede the natural selection of bad science

2019 ◽  
Author(s):  
Paul Smaldino ◽  
Matthew Adam Turner ◽  
Pablo Andrés Contreras Kallens
Keyword(s):  
Natural Selection ◽  
Peer Review ◽  
Strategic Behavior ◽  
Open Science ◽  
Negative Results ◽  
Methodological Rigor ◽  
Recent Developments ◽  
False Discoveries ◽  
Selection Of

Assessing scientists using exploitable metrics can lead to the degradation of research methods even without any strategic behavior on the part of individuals, via "the natural selection of bad science." Institutional incentives to maximize metrics like publication quantity and impact drive this dynamic. Removing these incentives is necessary, but institutional change is slow. However, recent developments suggest possible solutions with more rapid onsets. These include what we call open science improvements, which can reduce publication bias and improve the efficacy of peer review. In addition, there have been increasing calls for funders to move away from prestige- or innovation-based approaches in favor of lotteries. We investigated whether such changes are likely to improve the reproducibility of science even in the presence of persistent incentives for publication quantity through computational modeling. We found that modified lotteries, which allocate funding randomly among proposals that pass a threshold for methodological rigor, effectively reduce the rate of false discoveries, particularly when paired with open science improvements that increase the publication of negative results and improve the quality of peer review. In the absence of funding that targets rigor, open science improvements can still reduce false discoveries in the published literature but are less likely to improve the overall culture of research practices that underlie those publications.

scholarly journals The natural selection of bad science

2016 ◽  
Vol 3 (9) ◽  
pp. 160384 ◽  
Author(s):  
Paul E. Smaldino ◽  
Richard McElreath
Keyword(s):  
Natural Selection ◽  
Statistical Power ◽  
Meta Analysis ◽  
Computational Modelling ◽  
False Discovery Rates ◽  
Culture Of Science ◽  
Quality Of Research ◽  
Selection Of ◽  
High Output

Poor research design and data analysis encourage false-positive findings. Such poor methods persist despite perennial calls for improvement, suggesting that they result from something more than just misunderstanding. The persistence of poor methods results partly from incentives that favour them, leading to the natural selection of bad science. This dynamic requires no conscious strategizing—no deliberate cheating nor loafing—by scientists, only that publication is a principal factor for career advancement. Some normative methods of analysis have almost certainly been selected to further publication instead of discovery. In order to improve the culture of science, a shift must be made away from correcting misunderstandings and towards rewarding understanding. We support this argument with empirical evidence and computational modelling. We first present a 60-year meta-analysis of statistical power in the behavioural sciences and show that power has not improved despite repeated demonstrations of the necessity of increasing power. To demonstrate the logical consequences of structural incentives, we then present a dynamic model of scientific communities in which competing laboratories investigate novel or previously published hypotheses using culturally transmitted research methods. As in the real world, successful labs produce more ‘progeny,’ such that their methods are more often copied and their students are more likely to start labs of their own. Selection for high output leads to poorer methods and increasingly high false discovery rates. We additionally show that replication slows but does not stop the process of methodological deterioration. Improving the quality of research requires change at the institutional level.

scholarly journals Enhancing scientific dissemination in neuroscience via preprint peer-review: “Peer Community In Circuit Neuroscience”

2020 ◽  
Vol 2 ◽  
pp. e9
Author(s):  
Marion Mercier ◽  
Vincent Magloire ◽  
Mahesh Karnani
Keyword(s):  
Peer Review ◽  
New Technologies ◽  
Unmet Need ◽  
Open Science ◽  
Biomedical Science ◽  
Rigorous Evaluation ◽  
New Era ◽  
Negative Results ◽  
Scientific Results ◽  
Peer Community

The dissemination of scientific results and new technologies in biomedical science is rapidly evolving from an exclusive and fee-oriented publishing system towards more open, free and independent strategies for sharing knowledge. In this context, preprint servers such as bioRxiv answer a very real scientific need by enabling the rapid, free and easy dissemination of findings, regardless of whether these are novel, replicated, or even showcasing negative results. Currently, thousands of manuscripts are being shared via bioRxiv each month, and neuroscience is the largest and fastest growing subject category. However, commenting on bioRxiv is declining and no structured scientific validation such as peer-review is currently available. The Peer Community In (PCI) platform addresses this unmet need by facilitating the rigorous evaluation and validation of preprints, and PCI Circuit Neuroscience (PCI C Neuro) aims to develop and extend this tool for the neuroscience community. Here we discuss PCI C Neuro’s mission, how it works, and why it is an essential initiative in this new era of open science.

scholarly journals CODECHECK: an Open Science initiative for the independent execution of computations underlying research articles during peer review to improve reproducibility

F1000Research ◽  
2021 ◽  
Vol 10 ◽  
pp. 253
Author(s):  
Daniel Nüst ◽  
Stephen J. Eglen
Keyword(s):  
Peer Review ◽  
Building Block ◽  
Scientific Paper ◽  
Open Science ◽  
Research Articles ◽  
Scientific Publications ◽  
Multiple Dimensions ◽  
Computational Research ◽  
Textual Research

The traditional scientific paper falls short of effectively communicating computational research.  To help improve this situation, we propose a system by which the computational workflows underlying research articles are checked. The CODECHECK system uses open infrastructure and tools and can be integrated into review and publication processes in multiple ways. We describe these integrations along multiple dimensions (importance, who, openness, when). In collaboration with academic publishers and conferences, we demonstrate CODECHECK with 25 reproductions of diverse scientific publications. These CODECHECKs show that asking for reproducible workflows during a collaborative review can effectively improve executability. While CODECHECK has clear limitations, it may represent a building block in Open Science and publishing ecosystems for improving the reproducibility, appreciation, and, potentially, the quality of non-textual research artefacts. The CODECHECK website can be accessed here: https://codecheck.org.uk/.

scholarly journals Promoting Open Science to Increase the Trustworthiness of Evidence in Special Education

2018 ◽  
Vol 85 (1) ◽  
pp. 104-118 ◽  
Author(s):  
Bryan G. Cook ◽  
John Wills Lloyd ◽  
David Mellor ◽  
Brian A. Nosek ◽  
William J. Therrien
Keyword(s):  
Special Education ◽  
Scientific Evidence ◽  
Open Science ◽  
Cultural Norms ◽  
Policy And Practice ◽  
Individuals With Disabilities ◽  
Science Practices ◽  
Quality Of Research ◽  
Selection Of

Scientific evidence should guide the selection of practice for individuals with disabilities. Scientific evidence, however, must be trustworthy to move special education toward greater empirical certainty and more effective policies and practices. Transparency, openness, and reproducibility increase the trustworthiness of evidence. We propose that researchers in special education adopt emerging open-science reforms, such as preprints, data and materials sharing, preregistration of studies and analysis plans, and Registered Reports. Adoption of these practices will require shifts in cultural norms, guidelines, and incentives. We discuss how adopting open-science practices can advance the quality of research and, consequently, policy and practice in special education.

scholarly journals CODECHECK: an Open Science initiative for the independent execution of computations underlying research articles during peer review to improve reproducibility

F1000Research ◽  
2021 ◽  
Vol 10 ◽  
pp. 253
Author(s):  
Daniel Nüst ◽  
Stephen J. Eglen
Keyword(s):  
Peer Review ◽  
Building Block ◽  
Scientific Paper ◽  
Open Science ◽  
Research Articles ◽  
Scientific Publications ◽  
Multiple Dimensions ◽  
Computational Research ◽  
Textual Research

The traditional scientific paper falls short of effectively communicating computational research.  To help improve this situation, we propose a system by which the computational workflows underlying research articles are checked. The CODECHECK system uses open infrastructure and tools and can be integrated into review and publication processes in multiple ways. We describe these integrations along multiple dimensions (importance, who, openness, when). In collaboration with academic publishers and conferences, we demonstrate CODECHECK with 25 reproductions of diverse scientific publications. These CODECHECKs show that asking for reproducible workflows during a collaborative review can effectively improve executability. While CODECHECK has clear limitations, it may represent a building block in Open Science and publishing ecosystems for improving the reproducibility, appreciation, and, potentially, the quality of non-textual research artefacts. The CODECHECK website can be accessed here: https://codecheck.org.uk/.

Fast Growing Firms ("Gazelles") and the Evolution of Russian Economy

2007 ◽  
pp. 85-100 ◽  
Author(s):  
A. Yudanov
Keyword(s):  
Natural Selection ◽  
Volterra Model ◽  
High Quality ◽  
Fast Growing ◽  
Russian Economy ◽  
Exponential Trend ◽  
Important Addition ◽  
Quality Of Approximation ◽  
Selection Of

The article is based on the empirical study of fast growing firms ("Gazelles") in Russia. Sales volumes of many such firms are growing in accordance with the precise exponential trend with surprisingly high quality of approximation. The author finds strong links between this phenomenon and the concentration of most of "Gazelles" in market niches where demand limitations are practically non-existent. Conscious, purposeful entrepreneurial search of free niches becomes under described conditions an important addition to the classic mechanism of evolution of the economy through the natural selection of random changes. The author’s approach to these processes is based on a modification of the well known Lotka-Volterra model.

scholarly journals Peer review declaration

2021 ◽  
Vol 2057 (1) ◽  
pp. 011002
Keyword(s):  
Peer Review ◽  
Review Process ◽  
Acceptance Rate ◽  
Double Blind ◽  
Conference Series ◽  
Academy Of Sciences ◽  
Scientific Standards ◽  
Single Blind ◽  
Selection Of

All papers published in this volume of Journal of Physics: Conference Series have been peer reviewed through processes administered by the proceedings Editors. Reviews were conducted by expert referees to the professional and scientific standards expected of a proceedings journal published by IOP Publishing. Professional translators took part in the preparation of the articles to ensure the good quality of the language. All articles received professional proofreading, in accordance with the requirements of the publisher. The selection of papers was made on the basis of conference abstracts. Total number of contributions received: 275. On the recommendation of the organizing committee, the most significant contributions were proposed for publication in the Journal of Physics: Conference Series. • Type of peer review: Single-blind / Double-blind / Triple-blind / Open / Other (please describe): Double-blind for most articles and Triple-blind for articles with ambiguous reviews • Conference submission management system: ConfTool Pro 2.6.137 • Number of submissions received: 142 • Number of submissions sent for review: 142 • Number of submissions accepted: 142 • Acceptance Rate (Number of Submissions Accepted / Number of Submissions Received X 100): 100 • Average number of reviews per paper: 2.05 • Total number of reviewers involved: 21 • Any additional info on review process: only recommended by the organizing committee contributions were proposed for reviewing and publication in the Journal of Physics: Conference Series. • Contact person for queries: Maksim Sergeyevich Makarov, Kutateladze Institute of Thermophysics of the Siberian Branch of the Russian Academy of Sciences, [email protected]

scholarly journals Applying organized skepticism to preprints

2019 ◽  
Author(s):  
Howard I. Browman
Keyword(s):  
Peer Review ◽  
Low Cost ◽  
Scholarly Work ◽  
Negative Results ◽  
And Mathematics ◽  
The Many

A preprint is a scholarly work posted by the author(s) in an openly accessible platform, usually before or in parallel with submission to a peer reviewed publication. While the sharing of manuscripts via preprint platforms has been common for many years in disciplines such as physics and mathematics, uptake in other disciplines had, until very recently, been low. The use of preprints is now growing exponentially. The reasons for this appear to be based on several purported advantages of preprints that have not been thoroughly assessed: low cost (but is it sustainable?); immediate publication; establishes precedence; improves quality of papers pre-submission; eliminates journal hierarchies and inequalities; identifies hypotheses not to test further (by making negative results available); a source of submissions for journals. I will take a skeptical view of these advantages and ask: do preprints really accomplish all of this? I will also present the many disadvantages – even dangers – of preprints as these have not been adequately scrutinized: bypasses peer review allowing documents containing unvetted claims to be made freely available; results in multiple competing versions (all of which are citable) of what – without careful and informed examination – appears to be the same content to persist in perpetuity (the preprint version of which is much more likely to contain errors and unvetted claims); presently, no one is responsible for updating the preprint server version, nor to link it to the final published version.

scholarly journals KEBERADAAN SALMONELLA SP. PADA SUSU OLAHAN ASAL KEDAI SUSU DI SEKITAR PERMUKIMAN MAHASISWA INSTITUT PERTANIAN BOGOR

2020 ◽  
Vol 8 (1) ◽  
pp. 34-42
Author(s):  
Ulfa Apriliana Annisa ◽  
Mirnawati B Sudarwanto ◽  
Susi Soviana ◽  
Herwin Pisestyani
Keyword(s):  
Healthy Food ◽  
National Standard ◽  
Serological Tests ◽  
Selective Media ◽  
Settlement Area ◽  
Milk Samples ◽  
Negative Results ◽  
Selection Of

Nowadays, milk is one of the favorite drinks for Indonesian people. This relates to the awareness of society about healthy food is increasing. At the same time, there are many milk retail that are appearing. The purpose of this study was to detect the presence and level of contamination of Salmonella sp. in processed milk from milk retail around the students settlement area of ​​IPB University. In Indonesia the standard for identifying Salmonella sp. refer to Indonesian National Standard 2897: 2008. This method consists of 5 stages: pre-enrichment, enrichment, selective media planting, confirmation based on biochemical and serological tests. Milk samples in this study came from 14 milk retail in the students settlement area of the IPB University. The milk samples were taken are plain milk (milk that has not added flavor (flavorings), sugar, and ice). The selection of milk retail is taken within a radius of 2 km from the midpoint of the IPB campus in the Dramaga and Cilibende areas. Based on the results of this study, it was found that all milk samples examined showed negative results which means no Salmonella sp. contamination was found. From this study it can be concluded that the quality of processed milk sold in milk retail around the student settlement area of IPB University is good.

Sign in / Sign up

Export Citation Format

Share Document