Are questionable research practices facilitating new discoveries in sport and exercise medicine? The proportion of supported hypotheses is implausibly high

2020 ◽  
Vol 54 (22) ◽  
pp. 1365-1371
Author(s):  
Fionn Büttner ◽  
Elaine Toomey ◽  
Shane McClean ◽  
Mark Roe ◽  
Eamonn Delahunt
Keyword(s):  
Sports Medicine ◽  
Open Science ◽  
Original Research ◽  
Research Practices ◽  
Questionable Research Practices ◽  
Exercise Medicine ◽  
Study Results ◽  
Primary Hypothesis ◽  
Sport And Exercise ◽  
Study Hypothesis

Questionable research practices (QRPs) are intentional and unintentional practices that can occur when designing, conducting, analysing, and reporting research, producing biased study results. Sport and exercise medicine (SEM) research is vulnerable to the same QRPs that pervade the biomedical and psychological sciences, producing false-positive results and inflated effect sizes. Approximately 90% of biomedical research reports supported study hypotheses, provoking suspicion about the field-wide presence of systematic biases to facilitate study findings that confirm researchers’ expectations. In this education review, we introduce three common QRPs (ie, HARKing, P-hacking and Cherry-picking), perform a cross-sectional study to assess the proportion of original SEM research that reports supported study hypotheses, and draw attention to existing solutions and resources to overcome QRPs that manifest in exploratory research. We hypothesised that ≥ 85% of original SEM research studies would report supported study hypotheses. Two independent assessors systematically identified, screened, included, and extracted study data from original research articles published between 1 January 2019 and 31 May 2019 in the British Journal of Sports Medicine, Sports Medicine, the American Journal of Sports Medicine, and the Journal of Orthopaedic & Sports Physical Therapy. We extracted data relating to whether studies reported that the primary hypothesis was supported or rejected by the results. Study hypotheses, methodologies, and analysis plans were preregistered at the Open Science Framework. One hundred and twenty-nine original research studies reported at least one study hypothesis, of which 106 (82.2%) reported hypotheses that were supported by study results. Of 106 studies reporting that primary hypotheses were supported by study results, 75 (70.8%) studies reported that the primary hypothesis was fully supported by study results. The primary study hypothesis was partially supported by study results in 28 (26.4%) studies. We detail open science practices and resources that aim to safe-guard against QRPs that bely the credibility and replicability of original research findings.

scholarly journals Fascial tissue research in sports medicine: from molecules to tissue adaptation, injury and diagnostics: consensus statement

2018 ◽  
Vol 52 (23) ◽  
pp. 1497-1497 ◽  
Author(s):  
Martina Zügel ◽  
Constantinos N Maganaris ◽  
Jan Wilke ◽  
Karin Jurkat-Rott ◽  
Werner Klingler ◽  
...  
Keyword(s):  
Sports Medicine ◽  
High Performance ◽  
Consensus Statement ◽  
Exercise Physiology ◽  
Three Dimensional ◽  
Significant Loss ◽  
The Body ◽  
Exercise Medicine ◽  
Recreational Exercise ◽  
Sport And Exercise

The fascial system builds a three-dimensional continuum of soft, collagen-containing, loose and dense fibrous connective tissue that permeates the body and enables all body systems to operate in an integrated manner. Injuries to the fascial system cause a significant loss of performance in recreational exercise as well as high-performance sports, and could have a potential role in the development and perpetuation of musculoskeletal disorders, including lower back pain. Fascial tissues deserve more detailed attention in the field of sports medicine. A better understanding of their adaptation dynamics to mechanical loading as well as to biochemical conditions promises valuable improvements in terms of injury prevention, athletic performance and sports-related rehabilitation. This consensus statement reflects the state of knowledge regarding the role of fascial tissues in the discipline of sports medicine. It aims to (1) provide an overview of the contemporary state of knowledge regarding the fascial system from the microlevel (molecular and cellular responses) to the macrolevel (mechanical properties), (2) summarise the responses of the fascial system to altered loading (physical exercise), to injury and other physiological challenges including ageing, (3) outline the methods available to study the fascial system, and (4) highlight the contemporary view of interventions that target fascial tissue in sport and exercise medicine. Advancing this field will require a coordinated effort of researchers and clinicians combining mechanobiology, exercise physiology and improved assessment technologies.

scholarly journals The devil is in the details: Reporting and transparent research practices in sports medicine and orthopedic clinical trials

2021 ◽  
Author(s):  
Robert Schulz ◽  
Georg Langen ◽  
Robert Prill ◽  
Michael Cassel ◽  
Tracey Weissgerber
Keyword(s):  
Clinical Trial ◽  
Clinical Trials ◽  
Sports Medicine ◽  
Open Data ◽  
Open Science ◽  
Research Practices ◽  
Scientific Rigor ◽  
Sample Size Calculations ◽  
Research Findings ◽  
The Devil

Introduction: While transparent reporting of clinical trials is essential to assess the risk of bias and translate research findings into clinical practice, earlier studies have shown that deficiencies are common. This study examined current clinical trial reporting and transparent research practices in sports medicine and orthopedics. Methods: The sample included clinical trials published in the top 25% of sports medicine and orthopedics journals over eight months. Two independent reviewers assessed pre-registration, open data and criteria related to scientific rigor, the study sample, and data analysis. Results: The sample included 163 clinical trials from 27 journals. While the majority of trials mentioned rigor criteria, essential details were often missing. Sixty percent (confidence interval [CI] 53-68%) of trials reported sample size calculations, but only 32% (CI 25-39%) justified the expected effect size. Few trials indicated the blinding status of all main stakeholders (4%; CI 1-7%). Only 18% (CI 12-24%) included information on randomization type, method, and concealed allocation. Most trials reported participants' sex/gender (95%; CI 92-98%) and information on inclusion and exclusion criteria (78%; CI 72-84%). Only 20% (CI 14-26%) of trials were pre-registered. No trials deposited data in open repositories. Conclusions: These results will aid the sports medicine and orthopedics community in developing tailored interventions to improve reporting. While authors typically mention blinding, randomization and other factors, essential details are often missing. Greater acceptance of open science practices, like pre-registration and open data, is needed. These practices have been widely encouraged, we discuss systemic interventions that may improve clinical trial reporting. Registration: https://doi.org/10.17605/OSF.IO/9648H

scholarly journals ­Questionable Research Practices and Open Science in Quantitative Criminology

2021 ◽  
Author(s):  
Jason Chin ◽  
Justin T. Pickett ◽  
Simine Vazire ◽  
Alex O. Holcombe
Keyword(s):  
Open Science ◽  
Research Practices ◽  
Questionable Research Practices

scholarly journals Moving Sport and Exercise Science Forward: A Call for the Adoption of More Transparent Research Practices

2019 ◽  
Author(s):  
Aaron R Caldwell ◽  
Andrew David Vigotsky ◽  
Greg Nuckols ◽  
Ian Boardley ◽  
Julia Schmidt ◽  
...  
Keyword(s):  
Data Collection ◽  
Science Research ◽  
Exercise Science ◽  
Research Practices ◽  
Questionable Research Practices ◽  
Data Collection And Analysis ◽  
Primary Means ◽  
Sport And Exercise ◽  
Stage 1

The primary means for disseminating sport and exercise science research is currently through journal articles. However, not all studies, especially those with null findings, make it to formal publication. This publication bias towards positive findings may contribute to questionable research practices. Preregistration is a solution to prevent the publication of distorted evidence resulting from this system. This process asks authors to register their hypotheses and methods before data collection on a publicly available repository or by submitting a Registered Report. In the Registered Reports format, authors submit a Stage 1 manuscript to a participating journal that includes an introduction, methods, and any pilot data indicating the exploratory or confirmatory nature of the study. After a Stage 1 peer review, the manuscript can then be offered in-principle acceptance, rejected, or sent back for revisions to improve the quality of the study. If accepted, the project is guaranteed publication, assuming the authors follow the data collection and analysis protocol. After data collection, authors re-submit a Stage 2 manuscript that includes the results and discussion, and the study is evaluated on clarity and conformity with the planned analysis. In its final form, Registered Reports appear almost identical to a typical publication, but give readers confidence that the hypotheses and main analyses are less susceptible to bias from questionable research practices. From this perspective, we argue that inclusion of Registered Reports by researchers and journals will improve the transparency, replicability, and trust in sport and exercise science research.

scholarly journals Research practices and statistical reporting quality in 250 economic psychology master's theses: a meta-research investigation

2019 ◽  
Vol 6 (12) ◽  
pp. 190738 ◽  
Author(s):  
Jerome Olsen ◽  
Johanna Mosen ◽  
Martin Voracek ◽  
Erich Kirchler
Keyword(s):  
Statistical Power ◽  
Testing Effect ◽  
Reporting Quality ◽  
Open Science ◽  
Sample Size Determination ◽  
Effect Sizes ◽  
Research Practices ◽  
Questionable Research Practices ◽  
Standardized Effect Sizes ◽  
Research Findings

The replicability of research findings has recently been disputed across multiple scientific disciplines. In constructive reaction, the research culture in psychology is facing fundamental changes, but investigations of research practices that led to these improvements have almost exclusively focused on academic researchers. By contrast, we investigated the statistical reporting quality and selected indicators of questionable research practices (QRPs) in psychology students' master's theses. In a total of 250 theses, we investigated utilization and magnitude of standardized effect sizes, along with statistical power, the consistency and completeness of reported results, and possible indications of p -hacking and further testing. Effect sizes were reported for 36% of focal tests (median r = 0.19), and only a single formal power analysis was reported for sample size determination (median observed power 1 − β = 0.67). Statcheck revealed inconsistent p -values in 18% of cases, while 2% led to decision errors. There were no clear indications of p -hacking or further testing. We discuss our findings in the light of promoting open science standards in teaching and student supervision.

Early-Career Researchers’ Perceptions of the Prevalence of Questionable Research Practices, Potential Causes, and Open Science

2017 ◽  
Vol 48 (6) ◽  
pp. 365-371 ◽  
Author(s):  
Stefan Stürmer ◽  
Aileen Oeberst ◽  
Roman Trötschel ◽  
Oliver Decker
Keyword(s):  
Open Science ◽  
Early Career ◽  
Research Practice ◽  
Research Culture ◽  
Research Practices ◽  
Questionable Research Practices ◽  
Early Career Researchers ◽  
Incentive Structures ◽  
Exploratory Survey ◽  
The Future

Abstract. Young researchers of today will shape the field in the future. In light of current debates about social psychology’s research culture, this exploratory survey assessed early-career researchers’ beliefs (N = 88) about the prevalence of questionable research practices (QRPs), potential causes, and open science as a possible solution. While there was relative consensus that outright fraud is an exception, a majority of participants believed that some QRPs are moderately to highly prevalent what they attributed primarily to academic incentive structures. A majority of participants felt that open science is necessary to improve research practice. They indicated to consider some open science recommendations in the future, but they also indicated some reluctance. Limitation and implications of these findings are discussed.

scholarly journals Delphi developed syllabus for the medical specialty of sport and exercise medicine: part 2

2020 ◽  
Vol 55 (2) ◽  
pp. 81-83
Author(s):  
David Humphries ◽  
Rod Jaques ◽  
H Paul Dijkstra ◽  
Irfan Asif ◽  
Mark E Batt ◽  
...  
Keyword(s):  
Internal Medicine ◽  
Delphi Study ◽  
Sports Medicine ◽  
Medical Specialty ◽  
International Level ◽  
Exercise Medicine ◽  
Multiple Round ◽  
Sport And Exercise ◽  
Extensive Clinical Experience ◽  
Specific Learning

Training in the medical specialty of sport and exercise medicine (SEM) is available in many, but not all countries. In 2015, an independent Delphi group, the International Syllabus in Sport and Exercise Medicine Group (ISSEMG), was formed to create a basic syllabus for this medical specialty. The group provided the first part of this syllabus, by identifying 11 domains and a total of 80 general learning areas for the specialty, in December 2017. The next step in this process, and the aim of this paper was to determine the specific learning areas for each of the 80 general learning areas. A group of 26 physicians with a range of primary medical specialty qualifications including, Sport and Exercise Medicine, Family Medicine, Internal Medicine, Cardiology, Rheumatology and Anaesthetics were invited to participate in a multiple round online Delphi study to develop specific learning areas for each of the previously published general learning areas. All invitees have extensive clinical experience in the broader sports medicine field, and in one or more components of sports medicine governance at national and/or international level. SEM, Family Medicine, Internal Medicine, Cardiology, Rheumatology and Anaesthetics were invited to participate in a multiple round online Delphi study to develop specific learning areas for each of the previously published general learning areas. All invitees have extensive clinical experience in the broader sports medicine field, and in one or more components of sports medicine governance at national and/or international level. The hierarchical syllabus developed by the ISSEMG provides a useful resource in the planning, development and delivery of specialist training programmes in the medical specialty of SEM.

scholarly journals The Use of Questionable Research Practices to Survive in Academia Examined With Expert Elicitation, Prior-Data Conflicts, Bayes Factors for Replication Effects, and the Bayes Truth Serum

2021 ◽  
Vol 12 ◽  
Author(s):  
Rens van de Schoot ◽  
Sonja D. Winter ◽  
Elian Griffioen ◽  
Stephan Grimmelikhuijsen ◽  
Ingrid Arts ◽  
...  
Keyword(s):  
Bayesian Methods ◽  
Open Science ◽  
Expert Elicitation ◽  
Bayes Factors ◽  
Published Data ◽  
Research Practices ◽  
Academic Leaders ◽  
Questionable Research Practices ◽  
Unethical Research ◽  
Science Framework

The popularity and use of Bayesian methods have increased across many research domains. The current article demonstrates how some less familiar Bayesian methods can be used. Specifically, we applied expert elicitation, testing for prior-data conflicts, the Bayesian Truth Serum, and testing for replication effects via Bayes Factors in a series of four studies investigating the use of questionable research practices (QRPs). Scientifically fraudulent or unethical research practices have caused quite a stir in academia and beyond. Improving science starts with educating Ph.D. candidates: the scholars of tomorrow. In four studies concerning 765 Ph.D. candidates, we investigate whether Ph.D. candidates can differentiate between ethical and unethical or even fraudulent research practices. We probed the Ph.D.s’ willingness to publish research from such practices and tested whether this is influenced by (un)ethical behavior pressure from supervisors or peers. Furthermore, 36 academic leaders (deans, vice-deans, and heads of research) were interviewed and asked to predict what Ph.D.s would answer for different vignettes. Our study shows, and replicates, that some Ph.D. candidates are willing to publish results deriving from even blatant fraudulent behavior–data fabrication. Additionally, some academic leaders underestimated this behavior, which is alarming. Academic leaders have to keep in mind that Ph.D. candidates can be under more pressure than they realize and might be susceptible to using QRPs. As an inspiring example and to encourage others to make their Bayesian work reproducible, we published data, annotated scripts, and detailed output on the Open Science Framework (OSF).

Sign in / Sign up

Export Citation Format

Share Document