Lack of Reproducibility in Addiction Medicine

2019 ◽  
Author(s):  
Mopileola Tomi Adewumi ◽  
Nam Vo ◽  
Daniel Tritz ◽  
Jason Beaman ◽  
Matt Vassar
Keyword(s):  
Conflicts Of Interest ◽  
Data Extraction ◽  
Research Integrity ◽  
Data Availability ◽  
Addiction Medicine ◽  
Cross Sectional ◽  
Funding Sources ◽  
Current State ◽  
Meta Analyses ◽  
Substance Related Disorders

Background and aims: Credible research emphasizes transparency, openness, and reproducibility. These characteristics are fundamental to promoting and maintaining research integrity. This aim of this study was to evaluate the current state of transparency and reproducibility in the field of addiction science. Design: Cross-sectional designMeasurements: This study used the National Library of Medicine catalog to search for all journals using the subject terms tag: Substance-Related Disorders [ST]. Journals were then searched via PubMed in the timeframe of January 1, 2014 to December 31, 2018 and 300 publications were randomly selected. A pilot-tested Google form containing reproducibility/transparency characteristics was used for data extraction by two investigators who performed this process in a duplicate and blinded fashion. Findings: Slightly more than half of the publications were open access (152/293, 50.7%). Few publications had pre-registration (7/244, 2.87%), material availability (2/237, 1.23%), protocol availability (3/244 ,0.80%), data availability (28/244, 11.48%), and analysis script availability (2/244, 0.82%). Most publications provided a conflict of interest statement (221/293, 75.42%) and funding sources (268/293, 91.47%). One replication study was reported (1/244, 0.04%). Few publications were cited (64/238, 26.89%) and 0 were excluded from meta-analyses and/or systematic reviews.Conclusion: Our study found that current practices that promote transparency and reproducibility are lacking, and thus, there is much room for improvement. First, investigators should preregister studies prior to commencement. Researchers should also make the materials, data, analysis script publicly available. To foster reproducibility, individuals should remain transparent about funding sources for the project and financial conflicts of interest. Research stakeholders should work together toward improved solutions on these matters. With these protections in place, the field of addiction medicine can lead in dissemination of information necessary to treat patients.

scholarly journals Reporting of drug trial funding sources and author financial conflicts of interest in Cochrane and non-Cochrane meta-analyses: a cross-sectional study

BMJ Open ◽  
2020 ◽  
Vol 10 (5) ◽  
pp. e035633
Author(s):  
Kimberly Turner ◽  
Andrea Carboni-Jiménez ◽  
Carla Benea ◽  
Katharine Elder ◽  
Brooke Levis ◽  
...  
Keyword(s):  
Conflicts Of Interest ◽  
Multivariable Analysis ◽  
Cross Sectional Study ◽  
Sectional Study ◽  
Eligibility Criteria ◽  
Cross Sectional ◽  
Funding Sources ◽  
Cochrane Reviews ◽  
Randomised Controlled ◽  
Meta Analyses

ObjectiveTo (1) investigate the extent to which recently published meta-analyses report trial funding, author–industry financial ties and author–industry employment from included randomised controlled trials (RCTs), comparing Cochrane and non-Cochrane meta-analyses; (2) examine characteristics of meta-analyses independently associated with reporting funding sources of included RCTs; and (3) compare reporting among recently published Cochrane meta-analyses to Cochrane reviews published in 2010.DesignReview of consecutive sample of recently published meta-analyses.Data sourcesMEDLINE database via PubMed searched on 19 October 2018.Eligibility criteria for selecting articlesWe selected the 250 most recent meta-analyses listed in PubMed that included a documented search of at least one database, statistically combined results from ≥2 RCTs and evaluated the effects of a drug or class of drugs.Results90 of 107 (84%) Cochrane meta-analyses reported funding sources for some or all included trials compared with 21 of 143 (15%) non-Cochrane meta-analyses, a difference of 69% (95% CI 59% to 77%). Percent reporting was also higher for Cochrane meta-analyses compared with non-Cochrane meta-analyses for trial author–industry financial ties (44% versus 1%; 95% CI for difference 33% to 52%) and employment (17% versus 1%; 95% CI for difference 9% to 24%). In multivariable analysis, compared with Cochrane meta-analyses, the odds ratio (OR) for reporting trial funding was ≤0.11 for all other journal category and impact factor combinations. Compared with Cochrane reviews from 2010, reporting of funding sources of included RCTs among recently published Cochrane meta-analyses improved by 54% (95% CI 42% to 63%), and reporting of trial author–industry financial ties and employment improved by 37% (95% CI 26% to 47%) and 10% (95% CI 2% to 19%).ConclusionsReporting of trial funding sources, trial author–industry financial ties and trial author–industry employment in Cochrane meta-analyses has improved since 2010 and is higher than in non-Cochrane meta-analyses.

scholarly journals Usability of Mobile Health Apps for Postoperative Care: Systematic Review

10.2196/19099 ◽  
2020 ◽  
Vol 3 (2) ◽  
pp. e19099
Author(s):  
Ben Patel ◽  
Arron Thind
Keyword(s):  
Systematic Review ◽  
Systematic Reviews ◽  
Mobile Health ◽  
Conflicts Of Interest ◽  
Data Extraction ◽  
Measurement Tool ◽  
Validated Questionnaire ◽  
System Usability Scale ◽  
Meta Analyses ◽  
Mhealth Apps

Background Mobile health (mHealth) apps are increasingly used postoperatively to monitor, educate, and rehabilitate. The usability of mHealth apps is critical to their implementation. Objective This systematic review evaluates the (1) methodology of usability analyses, (2) domains of usability being assessed, and (3) results of usability analyses. Methods The A Measurement Tool to Assess Systematic Reviews checklist was consulted. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses reporting guideline was adhered to. Screening was undertaken by 2 independent reviewers. All included studies were assessed for risk of bias. Domains of usability were compared with the gold-standard mHealth App Usability Questionnaire (MAUQ). Results A total of 33 of 720 identified studies were included for data extraction. Of the 5 included randomized controlled trials (RCTs), usability was never the primary end point. Methodology of usability analyses included interview (10/33), self-created questionnaire (18/33), and validated questionnaire (9/33). Of the 3 domains of usability proposed in the MAUQ, satisfaction was assessed in 28 of the 33 studies, system information arrangement was assessed in 11 of the 33 studies, and usefulness was assessed in 18 of the 33 studies. Usability of mHealth apps was above industry average, with median System Usability Scale scores ranging from 76 to 95 out of 100. Conclusions Current analyses of mHealth app usability are substandard. RCTs are rare, and validated questionnaires are infrequently consulted. Of the 3 domains of usability, only satisfaction is regularly assessed. There is significant bias throughout the literature, particularly with regards to conflicts of interest. Future studies should adhere to the MAUQ to assess usability and improve the utility of mHealth apps.

scholarly journals Prenatal maternal stress and risk of neurodevelopmental disorders in the offspring: A systematic review and meta-analysis protocol

2018 ◽  
Vol 1 ◽  
pp. 15
Author(s):  
Nicla Manzari ◽  
Karen Matvienko-Sikar ◽  
Franco Baldoni ◽  
Gerard W. O'Keeffe ◽  
Ali S. Khashan
Keyword(s):  
Systematic Review ◽  
Maternal Stress ◽  
Data Extraction ◽  
Meta Analysis ◽  
Autism Spectrum ◽  
Prenatal Maternal Stress ◽  
Cross Sectional ◽  
Increased Risk ◽  
Meta Analyses ◽  
The Impact

Background: Prenatal maternal stress (PNMS) is defined as the experience of significant levels of prenatal stress, depression or anxiety during pregnancy. PNMS has been associated with increased risk of autism spectrum disorder (ASD) and attention-deficit hyperactivity disorder (ADHD) in exposed offspring. However, these findings are inconsistent and other studies found no association, meaning a clear consensus on the impact of PNMS on ASD and ADHD risk is required. The purpose of this systematic review and meta-analysis is to summarize and critically review the existing literature on the effects of PNMS on ASD and ADHD risk. Methods: Electronic databases (PubMed, PsycINFO, Web of Science, Scopus and EMBASE) will be searched for articles following a detailed search strategy. We will include cohort, case-control and cross-sectional studies that assessed maternal exposure to psychological and/or environmental stress and had ASD or ADHD as an outcome. Two reviewers will independently screen the titles, abstracts and full articles to identify eligible studies. We will use a standardised data extraction form for extracting data and a bias classification tool for assessing study quality. This systematic review will be reported according to the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA). The generic inverse variance method will be used if possible to perform meta-analyses. Ethics and dissemination: Ethical approval is not required for this study because it will not involve the conduct or inclusion of any experimental or personal data that would require informed consent.  The systematic review will be disseminated in peer-reviewed journals. PROSPERO registration number: CRD42018084222.

scholarly journals Usability of Mobile Health Apps for Postoperative Care: Systematic Review (Preprint)

2020 ◽  
Author(s):  
Ben Patel ◽  
Arron Thind
Keyword(s):  
Systematic Review ◽  
Systematic Reviews ◽  
Mobile Health ◽  
Conflicts Of Interest ◽  
Data Extraction ◽  
Measurement Tool ◽  
Validated Questionnaire ◽  
System Usability Scale ◽  
Meta Analyses ◽  
Mhealth Apps

BACKGROUND Mobile health (mHealth) apps are increasingly used postoperatively to monitor, educate, and rehabilitate. The usability of mHealth apps is critical to their implementation. OBJECTIVE This systematic review evaluates the (1) methodology of usability analyses, (2) domains of usability being assessed, and (3) results of usability analyses. METHODS The A Measurement Tool to Assess Systematic Reviews checklist was consulted. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses reporting guideline was adhered to. Screening was undertaken by 2 independent reviewers. All included studies were assessed for risk of bias. Domains of usability were compared with the gold-standard mHealth App Usability Questionnaire (MAUQ). RESULTS A total of 33 of 720 identified studies were included for data extraction. Of the 5 included randomized controlled trials (RCTs), usability was never the primary end point. Methodology of usability analyses included interview (10/33), self-created questionnaire (18/33), and validated questionnaire (9/33). Of the 3 domains of usability proposed in the MAUQ, satisfaction was assessed in 28 of the 33 studies, system information arrangement was assessed in 11 of the 33 studies, and usefulness was assessed in 18 of the 33 studies. Usability of mHealth apps was above industry average, with median System Usability Scale scores ranging from 76 to 95 out of 100. CONCLUSIONS Current analyses of mHealth app usability are substandard. RCTs are rare, and validated questionnaires are infrequently consulted. Of the 3 domains of usability, only satisfaction is regularly assessed. There is significant bias throughout the literature, particularly with regards to conflicts of interest. Future studies should adhere to the MAUQ to assess usability and improve the utility of mHealth apps.

scholarly journals The relationship between maternal stress during pregnancy and up to two years after birth and risk of child overweight and obesity: Protocol for a Systematic Review

2019 ◽  
Vol 1 ◽  
pp. 24
Author(s):  
Jennifer Cooney ◽  
Caragh Flannery ◽  
Ali S. Khashan ◽  
Anja C. Huizink ◽  
Karen Matvienko-Sikar
Keyword(s):  
Systematic Review ◽  
Childhood Obesity ◽  
Maternal Stress ◽  
Data Extraction ◽  
Personal Data ◽  
Overweight And Obesity ◽  
Cochrane Library ◽  
Cross Sectional ◽  
Child Overweight ◽  
Meta Analyses

Background: Childhood obesity presents a significant public health challenge globally. The period from conception to two years after birth, the first 1000 days, represents a critical period during which the experience of maternal stress may be related to the development of childhood obesity.  Research to date suggests some positive associations between maternal stress during the first 1000 days and childhood obesity, but findings are inconsistent and have not yet been comprehensively synthesised. The purpose of this review is to systematically examine the association between maternal stress during the first 1000 days and the risk of child overweight and obesity. Methods: The following electronic databases will be searched from inception using a detailed search strategy: the Cochrane Library, MEDLINE, PsycINFO, EMBASE, CINAHL, Maternity and Infant Care, and Web of Science. Cohort, case-control, and cross-sectional studies examining maternal stress during the first 1000 days and child overweight and obesity up to the age of 10 years will be included. Titles, abstracts and full articles will be screened by two investigators independently to identify eligible studies. A standardised data extraction form will be used to extract data including: study design; maternal stress exposure; child outcome; exclusion criteria; participant characteristics; and assessment methods. The Cochrane Collaboration’s bias classification tool for observational studies will be used to assess study quality. This protocol is reported according to the Preferred Reporting Items for Systematic reviews and Meta-Analyses protocol (PRISMA-P) checklist, and the systematic review will be conducted and reported following the PRISMA checklist. If possible, random effects models will be used to perform meta-analyses. Ethics and dissemination: Ethical approval is not required for this study as it will not involve conducting experimental research, nor include identifying personal data.  The systematic review will be disseminated in peer-reviewed journals. PROSPERO registration number: CRD42018100363

scholarly journals O32: TRANSPARENCY IN SURGICAL RANDOMISED CONTROLLED TRIALS: CROSS-SECTIONAL, OBSERVATIONAL STUDY

2021 ◽  
Vol 108 (Supplement_1) ◽  
Author(s):  
JA Helliwell ◽  
B Shelton ◽  
H Mahmood ◽  
R Blanco-Colino ◽  
JE Fitzgerald ◽  
...  
Keyword(s):  
Data Sharing ◽  
Randomised Controlled Trials ◽  
Conflicts Of Interest ◽  
Trial Registration ◽  
Controlled Trials ◽  
Cross Sectional ◽  
Research Waste ◽  
Funding Sources ◽  
Randomised Controlled ◽  
Personal Conflicts Of Interest

Abstract Introduction Randomised controlled trials (RCT) often provide the scientific basis on which commissioning and treatment decisions are made. It is essential that their results and methods are reported transparently. The aim of this study was to explore transparency with respect to trial registration, disclosure of funding sources, conflicts of interest (COI), and data sharing. Method This was a cross-sectional review of surgical RCTs. Data were extracted from RCTs in ten high-impact journals published in the years 2009, 2012, 2015, and 2018. Outcomes of interest were the incidence of reported trial registration, disclosure of funding sources, disclosure of investigator COI, and presence of a statement of data sharing plans. Result A total of 475 were eligible for analysis. Trial registration was present in 73 (67%) studies in 2009, 137 (84%) in 2012, 111 (89%) in 2015 and 110 (93%) in 2018. Funding statements were provided in 55%, 65%, 69.4%, and 75.4% of manuscripts, respectively. Conflicts of interest statements were provided in 49.5%, 89.1%, 94.6%, and 98.3% of manuscripts, respectively. Data sharing statements were present in only 15 (3.2%) RCTs. Eleven of these were in studies published most recently in 2018. Conclusion Trial registration, presence of funding statements, and disclosure of personal conflicts of interest in surgical RCTs have improved rapidly over the last 10 years. In contrast, disclosure of data sharing plans is exceptionally low. This may contribute to research waste and represents an essential target for improvement. Take-home message Trial registration, presence of funding statements, and disclosure of personal conflicts of interest in surgical RCTs have improved rapidly over the last 10 years. In contrast, disclosure of data sharing plans is exceptionally low. This may contribute to research waste and represents an essential target for improvement.

scholarly journals Kidney Complications of COVID-19: A Systematic Review and Meta-Analysis

2021 ◽  
Vol 21 (1) ◽  
pp. e00503-e00503
Author(s):  
Naser Nasiri ◽  
Shoboo Rahmati ◽  
Abbas Etminan ◽  
Hamid Sharifi ◽  
Azam Bazrafshan ◽  
...  
Keyword(s):  
Systematic Review ◽  
Data Extraction ◽  
Meta Analysis ◽  
Kidney Injury ◽  
Process Data ◽  
Screening Process ◽  
Cross Sectional ◽  
Normal Ranges ◽  
Pooled Prevalence ◽  
Meta Analyses

Background: Some patients with coronavirus disease 2019 (COVID-19) have been reported to have developed mild to severe kidney injuries. The current systematic review and meta-analysis was carried out to estimate the prevalence and incidence of acute kidney injury (AKI) among COVID-19 patients. Study design: A systematic review and meta-analysis Methods: PubMed, Embase, Scopus, Web of Science, and MedRxiv databases were searched from December 1, 2019, up to July 27, 2020. Two independent co-authors completed the screening process, data extraction, and quality assessment of the retrieved records. Random-effects meta-analyses were used to determine the pooled prevalence and 95% confidence interval (CI) of AKI among COVID-19 patients. Results: Out of 2,332 unique identified records, 51 studies were included in the review. Overall, the studies were carried out on 25,600 patients. A total of 6,505 patients (in 18 cross-sectional studies) were included to estimate the pooled prevalence of AKI, and 18,934 patients (in 27 cohort studies) were included to determine the pooled incidence of AKI. The pooled prevalence of AKI was estimated as 10.08% (95% CI: 4.59, 17.32; I 2=98.56%; P<0.001). Furthermore, the pooled incidence of AKI was 12.78% (95% CI: 7.38, 19.36; I 2=99.27%; P<0.001). The mean (95% CI) values of serum creatinine (SCr), blood urea nitrogen (BUN), potassium, and sodium were 76.10 (69.36, 82.84), 4.60 (4.04, 5.30), 3.94 (3.78, 4.11), and 139.30 (138.26, 140.36) mmol/L, respectively. Conclusions: The AKI is a considerable complication among COVID-19 patients and should be screened for on clinical examinations. The BUN, SCr, potassium, and sodium levels were within the normal ranges.

scholarly journals Evaluering av medisinsk behandling: Hva vet vi, og hvordan bør kunnskapen håndteres?

2009 ◽  
Vol 18 (2) ◽  
Author(s):  
Jan Magnus Bjordal ◽  
Atle Klovning ◽  
Lars Slørdal
Keyword(s):  
Clinical Practice ◽  
Conflicts Of Interest ◽  
Meta Analysis ◽  
Quality Criteria ◽  
Level Of Evidence ◽  
Current State ◽  
Lower Back ◽  
Study Results ◽  
Meta Analyses ◽  
Therapeutic Alternatives

Data on the effects of medical therapies are available from clinical studies. Well planned and executed randomised, placebo-controlled studies are given the highest level of evidence when study results are reviewed. The available body of data on different therapies lend themselves to synthesis in clinical overviews or therapy recommendations that regularly achieve status as normative for clinical practice. This warrants caution and rigour as far as the underlying procedures are concerned. The current review encompasses a brief discussion of previous failures of pharmacovigilance, including the thalidomide disaster and the COX-2 debacle. The quality criteria for data evaluation and the methodological basis for meta-analyses are presented. Possible sources of error in meta-analyses, such as different reporting of effects, publication bias, heterogeneity, conflicts of interest and problems with evaluation of side effects, are discussed. The advantages of meta-analysis are exemplified with data on therapy of lower back pain, which recently has been subjected to clinical practice guidelines that, in our view, have several shortcomings. Meta-analyses have significant advantages over less formalised methods for the evaluation of the current state of knowledge of therapeutic alternatives.

scholarly journals Evaluation of Reproducibility in Urology Publications

2019 ◽  
Author(s):  
Shelby Lynn Rauh ◽  
Bradley S. Johnson ◽  
Aaron Bowers ◽  
Daniel Tritz ◽  
Benjamin Matthew Vassar
Keyword(s):  
Conflicts Of Interest ◽  
Data Availability ◽  
Research Quality ◽  
Raw Data ◽  
The Public ◽  
Availability Analysis ◽  
Multiple Indicators ◽  
Journal Editors ◽  
Funding Sources ◽  
Collaborative Efforts

Take Home MessageMany components of transparency and reproducibility are lacking in urology publications, making study replication, at best, difficult.IntroductionReproducibility is essential for the integrity of scientific research. Reproducibility is measured by the ability of investigators to replicate the outcomes of an original publication by using the same materials and procedures.MethodsWe sampled 300 publications in the field of urology for assessment of multiple indicators of reproducibility, including material availability, raw data availability, analysis script availability, pre-registration information, links to protocols, and whether the publication was freely available to the public. Publications were also assessed for statements about conflicts of interest and funding sources.ResultsOf the 300 sample publications, 171 contained empirical data and could be analyzed for reproducibility. Of the analyzed articles, 0.58% (1/171) provided links to protocols, and none of the studies provided analysis scripts. Additionally, 95.91% (164/171) did not provide accessible raw data, 97.53% (158/162) did not provide accessible materials, and 95.32% (163/171) did not state they were pre-registered.ConclusionCurrent urology research does not consistently provide the components needed to reproduce original studies. Collaborative efforts from investigators and journal editors are needed to improve research quality, while minimizing waste and patient risk.

scholarly journals Conflicts of interest among authors of systematic reviews and meta-analyses investigating interventions for melanoma: Cross-sectional Analysis (Preprint)

2020 ◽  
Author(s):  
Zane Rulon ◽  
Kalyn Powers ◽  
J. Michael Anderson ◽  
Michael Weaver ◽  
Austin Johnson ◽  
...  
Keyword(s):  
Treatment Group ◽  
Systematic Reviews ◽  
Conflicts Of Interest ◽  
The United States ◽  
Cross Sectional Study ◽  
Cross Sectional ◽  
Exact Test ◽  
Meta Analyses ◽  
Sectional Analysis ◽  
States Patent

BACKGROUND Background: Previous studies have highlighted the potential influence industry relationships may have on the outcomes of medical research. OBJECTIVE Objectives: We aimed to determine the prevalence of author COI in systematic reviews focusing on melanoma interventions, as well as determine whether the presence of these COI were associated with an increased likelihood of reporting favorable results and conclusions. METHODS Methods: This cross-sectional study included systematic reviews with or without meta-analyses focusing on interventions for melanoma. We searched MEDLINE and Embase for eligible systematic reviews published between September 1, 2016 and June 2, 2020. COI disclosures were cross-referenced with information from the CMS Open Payments Database, Dollars for Profs, Google Patents, the United States Patent and Trademark Office (USPTO), and previously published COI disclosure statements. Results were quantified using descriptive statistics and relationships were evaluated by Fisher's exact test. RESULTS Results: Of the 23 systematic reviews included in our sample, 12 (12/23; 52%) had at least one author with a COI. Of these reviews, seven (58%) reported narrative results favoring the treatment group and nine (75%) reported conclusions favoring the treatment group. Of the 11 systematic reviews without a conflicted author, four (36%) reported results favoring the treatment group and five (45%) reported conclusions favoring the treatment group. We found no significant association between the presence of author COI and the favorability of results (p= 0.53) or conclusions (p= 0.15). CONCLUSIONS Conclusions: Author COI did not appear to influence the outcomes of systematic reviews regarding melanoma interventions. Clinicians and other readers of dermatology literature should be cognizant of the influence that industry may have on the nature of reported outcomes, including those from systematic reviews and meta-analyses.

Sign in / Sign up

Export Citation Format

Share Document