Evaluation of Spin in the Abstracts of Systematic Reviews and Meta-Analyses Focused on the Treatment of Acne Vulgaris: Cross-Sectional Analysis

Background Spin is the misrepresentation of study findings, which may positively or negatively influence the reader’s interpretation of the results. Little is known regarding the prevalence of spin in abstracts of systematic reviews, specifically systematic reviews pertaining to the management and treatment of acne vulgaris. Objective The primary objective of this study was to characterize and determine the frequency of the most severe forms of spin in systematic review abstracts and to evaluate whether various study characteristics were associated with spin. Methods Using a cross-sectional study design, we searched PubMed and EMBASE for systematic reviews focusing on the management and treatment of acne vulgaris. Our search returned 316 studies, of which 36 were included in our final sample. To be included, each systematic review must have addressed either pharmacologic or nonpharmacologic treatment of acne vulgaris. These studies were screened, and data were extracted in duplicate by two blinded investigators. We analyzed systematic review abstracts for the nine most severe types of spin. Results Spin was present in 31% (11/36) of abstracts. A total of 12 examples of spin were identified in the 11 abstracts containing spin, with one abstract containing two instances of spin. The most common type of spin, selective reporting of or overemphasis on efficacy outcomes or analysis favoring the beneficial effect of the experimental intervention, was identified five times (5/12, 42%). A total of 44% (16/36) of studies did not report a risk of bias assessment. Of the 11 abstracts containing spin, six abstracts (55%) had not reported a risk of bias assessment or performed a risk of bias assessment but did not discuss it. Spin in abstracts was not significantly associated with a specific intervention type, funding source, or journal impact factor. Conclusions Spin is present in the abstracts of systematic reviews and meta-analyses covering the treatment of acne vulgaris. This paper raises awareness of spin in abstracts and emphasizes the importance of its recognition, which may lead to fewer incidences of spin in future studies.

Download Full-text

Evaluation of Spin in the Abstracts of Systematic Reviews and Meta-Analyses Focused on the Treatment of Acne Vulgaris: Cross-Sectional Analysis (Preprint)

10.2196/preprints.16978 ◽

2019 ◽

Author(s):

Ryan Ottwell ◽

Taylor C Rogers ◽

J Michael Anderson ◽

Austin Johnson ◽

Matt Vassar

Keyword(s):

Systematic Review ◽

Systematic Reviews ◽

Acne Vulgaris ◽

Cross Sectional Study ◽

Primary Objective ◽

Risk Of Bias ◽

Selective Reporting ◽

Cross Sectional ◽

Bias Assessment ◽

Meta Analyses

BACKGROUND Spin is the misrepresentation of study findings, which may positively or negatively influence the reader’s interpretation of the results. Little is known regarding the prevalence of spin in abstracts of systematic reviews, specifically systematic reviews pertaining to the management and treatment of acne vulgaris. OBJECTIVE The primary objective of this study was to characterize and determine the frequency of the most severe forms of spin in systematic review abstracts and to evaluate whether various study characteristics were associated with spin. METHODS Using a cross-sectional study design, we searched PubMed and EMBASE for systematic reviews focusing on the management and treatment of acne vulgaris. Our search returned 316 studies, of which 36 were included in our final sample. To be included, each systematic review must have addressed either pharmacologic or nonpharmacologic treatment of acne vulgaris. These studies were screened, and data were extracted in duplicate by two blinded investigators. We analyzed systematic review abstracts for the nine most severe types of spin. RESULTS Spin was present in 31% (11/36) of abstracts. A total of 12 examples of spin were identified in the 11 abstracts containing spin, with one abstract containing two instances of spin. The most common type of spin, <i>selective reporting of or overemphasis on efficacy outcomes or analysis favoring the beneficial effect of the experimental intervention,</i> was identified five times (5/12, 42%). A total of 44% (16/36) of studies did not report a risk of bias assessment. Of the 11 abstracts containing spin, six abstracts (55%) had not reported a risk of bias assessment or performed a risk of bias assessment but did not discuss it. Spin in abstracts was not significantly associated with a specific intervention type, funding source, or journal impact factor. CONCLUSIONS Spin is present in the abstracts of systematic reviews and meta-analyses covering the treatment of acne vulgaris. This paper raises awareness of spin in abstracts and emphasizes the importance of its recognition, which may lead to fewer incidences of spin in future studies.

Download Full-text

Methodological quality of systematic reviews on treatments for depression: a cross-sectional study

Epidemiology and Psychiatric Sciences ◽

10.1017/s2045796017000208 ◽

2017 ◽

Vol 27 (6) ◽

pp. 619-627 ◽

Cited By ~ 6

Author(s):

V. C. H. Chung ◽

X. Y. Wu ◽

Y. Feng ◽

R. S. T. Ho ◽

S. Y. S. Wong ◽

...

Keyword(s):

Systematic Reviews ◽

Methodological Quality ◽

A Priori ◽

Cross Sectional Study ◽

Risk Of Bias ◽

Sectional Study ◽

Pharmacological Treatments ◽

Cross Sectional ◽

Depression Treatments

Aims.Depression is one of the most common mental disorders and identifying effective treatment strategies is crucial for the control of depression. Well-conducted systematic reviews (SRs) and meta-analyses can provide the best evidence for supporting treatment decision-making. Nevertheless, the trustworthiness of conclusions can be limited by lack of methodological rigour. This study aims to assess the methodological quality of a representative sample of SRs on depression treatments.Methods.A cross-sectional study on the bibliographical and methodological characteristics of SRs published on depression treatments trials was conducted. Two electronic databases (the Cochrane Database of Systematic Reviews and the Database of Abstracts of Reviews of Effects) were searched for potential SRs. SRs with at least one meta-analysis on the effects of depression treatments were considered eligible. The methodological quality of included SRs was assessed using the validated AMSTAR (Assessing the Methodological Quality of Systematic Reviews) tool. The associations between bibliographical characteristics and scoring on AMSTAR items were analysed using logistic regression analysis.Results.A total of 358 SRs were included and appraised. Over half of included SRs (n = 195) focused on non-pharmacological treatments and harms were reported in 45.5% (n = 163) of all studies. Studies varied in methods and reporting practices: only 112 (31.3%) took the risk of bias among primary studies into account when formulating conclusions; 245 (68.4%) did not fully declare conflict of interests; 93 (26.0%) reported an ‘a priori’ design and 104 (29.1%) provided lists of both included and excluded studies. Results from regression analyses showed: more recent publications were more likely to report ‘a priori’ designs [adjusted odds ratio (AOR) 1.31, 95% confidence interval (CI) 1.09–1.57], to describe study characteristics fully (AOR 1.16, 95% CI 1.06–1.28), and to assess presence of publication bias (AOR 1.13, 95% CI 1.06–1.19), but were less likely to list both included and excluded studies (AOR 0.86, 95% CI 0.81–0.92). SRs published in journals with higher impact factor (AOR 1.14, 95% CI 1.04–1.25), completed by more review authors (AOR 1.12, 95% CI 1.01–1.24) and SRs on non-pharmacological treatments (AOR 1.62, 95% CI 1.01–2.59) were associated with better performance in publication bias assessment.Conclusion.The methodological quality of included SRs is disappointing. Future SRs should strive to improve rigour by considering of risk of bias when formulating conclusions, reporting conflict of interests and authors should explicitly describe harms. SR authors should also use appropriate methods to combine the results, prevent language and publication biases, and ensure timely updates.

Download Full-text

Selecting Risk of Bias Tools for Observational Studies for a Systematic Review of Anthropometric Measurements and Dental Caries among Children

International Journal of Environmental Research and Public Health ◽

10.3390/ijerph18168623 ◽

2021 ◽

Vol 18 (16) ◽

pp. 8623

Author(s):

Rokiah Mamikutty ◽

Ameera Syafiqah Aly ◽

Jamaludin Marhazlinda

Keyword(s):

Systematic Review ◽

Dental Caries ◽

Observational Studies ◽

Cross Sectional Study ◽

Risk Of Bias ◽

Anthropometric Measurements ◽

Sectional Study ◽

Kappa Score ◽

Cross Sectional ◽

Advantages And Disadvantages

In conducting a systematic review, assessing the risk of bias of the included studies is a vital step; thus, choosing the most pertinent risk of bias (ROB) tools is crucial. This paper determined the most appropriate ROB tools for assessing observational studies in a systematic review assessing the association between anthropometric measurements and dental caries among children. First, we determined the ROB tools used in previous reviews on a similar topic. Subsequently, we reviewed articles on ROB tools to identify the most recommended ROB tools for observational studies. Of the twelve ROB tools identified from the previous steps, three ROB tools that best fit the eight criteria of a good ROB tool were the Newcastle–Ottawa Scale (NOS) for cohort and case-control studies, and Agency for Healthcare Research and Quality (AHRQ) and the Effective Public Health Practice Project (EPHPP) for a cross-sectional study. We further assessed the inter-rater reliability for all three tools by analysing the percentage agreement, inter-class correlation coefficient (ICC) and kappa score. The overall percentage agreements and reliability scores of these tools ranged from good to excellent. Two ROB tools for the cross-sectional study were further evaluated qualitatively against nine of a tool’s advantages and disadvantages. Finally, the AHRQ and NOS were selected as the most appropriate ROB tool to assess cross-sectional and cohort studies in the present review.

Download Full-text

Use and reporting of risk of bias tools in 825 systematic reviews of acupuncture: a cross-sectional study

Acupuncture in Medicine ◽

10.1177/0964528420946043 ◽

2020 ◽

pp. 096452842094604

Author(s):

Youlin Long ◽

Xin Wang ◽

Wenzhe Xiao ◽

Rui Chen ◽

Qiong Guo ◽

...

Keyword(s):

Impact Factor ◽

Systematic Reviews ◽

English Language ◽

Chinese Language ◽

Descriptive Analysis ◽

Cross Sectional Study ◽

Risk Of Bias ◽

Cross Sectional ◽

Knowledge Infrastructure ◽

Chinese National Knowledge Infrastructure

Objective: To assess the use and reporting of risk of bias (RoB) tools in systematic reviews (SRs) of acupuncture. Study design and setting: We extracted and analyzed information relating to RoB in acupuncture SRs via Medline, Embase and the Chinese CNKI (Chinese National Knowledge Infrastructure), WanFang and VIP databases from their inception to 24 November 2017. Three subgroup analyses were used to check the influence of language, journal type and impact factor, following which we used descriptive analysis. Results: We included 825 acupuncture SRs, of which 48% used the Cochrane RoB tool. Only 36% used the latest version of the Cochrane Handbook (version 5.1.0 at time of writing) with higher proportions among Cochrane SRs (65%) versus non-Cochrane SRs (34%), and high impact factor journals (58%) versus low or no impact factor journals (28% and 38%, respectively). In the last decade, there were notable increases in the use of the Cochrane RoB tool and Cochrane Handbook version 5.1.0, of 43% and 19%, respectively. Chinese-language SRs demonstrated proportionally higher tendencies to report an incorrect Cochrane Handbook version, increasing by 14% in the last 5 years. Additionally, 7% SRs did not report any results, and only 10% reported relatively complete and adequate RoB assessment. Cochrane SRs reported more complete assessments than Chinese-language or non-Cochrane English-language SRs. Conclusion: Use and reporting of RoB tools were suboptimal. Proportionally, use of the Cochrane RoB tool and Cochrane Handbook version 5.1.0 was low but rising. Our results highlight the prevalence and concerns of using unsuitable tools and the issue of incomplete RoB reporting. RoB tool application requires further improvement.

Download Full-text

Faculty Opinions recommendation of Characteristics and quality of systematic reviews and meta-analyses of observational nutritional epidemiology: a cross-sectional study.

Faculty Opinions – Post-Publication Peer Review of the Biomedical Literature ◽

10.3410/f.739779843.793586563 ◽

2021 ◽

Author(s):

Saravana Kumar ◽

Esther Tian

Keyword(s):

Systematic Reviews ◽

Cross Sectional Study ◽

Nutritional Epidemiology ◽

Sectional Study ◽

Cross Sectional ◽

Meta Analyses

Download Full-text

P14.71 An assessment of the reporting and methodological quality of meningioma systematic reviews and meta-analyses

Neuro-Oncology ◽

10.1093/neuonc/noab180.178 ◽

2021 ◽

Vol 23 (Supplement_2) ◽

pp. ii51-ii52

Author(s):

A M George ◽

S Gupta ◽

S M Keshwara ◽

M A Mustafa ◽

C S Gillespie ◽

...

Keyword(s):

Systematic Review ◽

Systematic Reviews ◽

Methodological Quality ◽

Meta Analysis ◽

Neurofibromatosis Type ◽

Risk Of Bias ◽

Measurement Tool ◽

High Quality ◽

Meta Analyses

Abstract BACKGROUND Systematic reviews and meta-analyses constitute the highest level of research evidence and for a disease with limited clinical trial activity, are often relied upon to help inform clinical practice. This review of reviews evaluates both the reporting & methodological quality of meningioma evidence syntheses. MATERIAL AND METHODS Potentially eligible meningioma reviews published between 1st January 1990 and 31st December 2020 were identified from eight electronic databases. Inclusion required the study to meet the Cochrane guideline definition of a systematic review or meta-analysis. Reviews concerning neurofibromatosis type 2, spinal and pediatric meningiomas were excluded. The reporting and methodological quality of articles were assessed against the following modified guidelines: Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA), A MeaSurement Tool to Assess Systematic Reviews (AMSTAR2) and the Risk of Bias in Systematic Reviews (ROBIS) guidelines. RESULTS 117 systematic reviews were identified, 57 of which included meta-analysis (48.7%). The number of meningioma systematic reviews published each year has increased with 63 studies (53.9%) published between 01/2018 and 12/2020. A median of 17 studies (IQR 9–29) were included per review. Impact factor of journals publishing a systematic review with or without a meta-analysis was similar (median 2.3 vs 1.8, P=0.397). The mean PRISMA scores for systematic reviews with a meta-analysis was 21.11 (SD 4.1, 78% adherence) and without was 13.89 (SD 3.4, 63% adherence). Twenty-nine systematic reviews with meta-analysis (51%) and 11 without meta-analysis (18%) achieved greater than 80% adherence to PRISMA recommendations. Methodological quality assessment using AMSTAR2 revealed one study (0.9%) as high quality whilst 111 (94.8%) studies were graded as critically low. One hundred and two articles (87.2%) did not utilize a comprehensive search strategy as defined by the AMSTAR2 tool. Ninety-nine studies (84.6%) obtained a high level of concern for potential bias as per the ROBIS assessment. One hundred and eight articles (92.3%) failed to present information that a protocol had been established prior to study commencement and 76 articles (65.0%) did not conduct a risk of bias assessment. Across the three tools, domains relating to the establishment of a protocol prior to review commencement and conducting appropriate risk of bias assessments were frequently low scoring. CONCLUSION Overall reporting and methodological quality of meningioma systematic reviews was sub-optimal. Established critical appraisal tools and reporting guidelines should be utilized a priori to assist in producing high-quality systematic reviews.

Download Full-text

Spin in Abstracts of Systematic Reviews and Meta-Analyses of Melanoma Therapies: A Cross-Sectional Analysis (Preprint)

10.2196/preprints.33996 ◽

2021 ◽

Author(s):

Ross Nowlin ◽

Alexis Wirtz ◽

David Wenger ◽

Ryan Ottwell ◽

Courtney Cook ◽

...

Keyword(s):

Systematic Reviews ◽

Reference Group ◽

Human Subjects ◽

Meta Analysis ◽

Acne Vulgaris ◽

Inclusion Criteria ◽

Cross Sectional ◽

Study Characteristics ◽

Definition Of ◽

Meta Analyses

BACKGROUND Spin is defined as the misrepresentation of a study’s results, which may lead to misperceptions or misinterpretation of the findings. Spin has previously been found in randomized controlled trials and systematic reviews of acne vulgaris treatments and treatments of various non-dermatological conditions. OBJECTIVE The purpose of this study was to quantify the presence of spin in abstracts of systematic reviews and meta-analyses of melanoma therapies and identify any related secondary characteristics of these articles. METHODS e used a cross-sectional approach on June 2, 2020, to search the MEDLINE and Embase databases from their inception. To meet inclusion criteria, a study was required to be a systematic review or meta-analysis pertaining to the treatment of melanoma in human subjects, and reported in English. We used the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols (PRISMA) definition of systematic reviews and meta-analyses. Data were extracted in a masked, duplicate fashion. We conducted a powered bivariate linear regression and calculated odds ratios for each study characteristic. RESULTS A total of 200 systematic reviews met the inclusion criteria. We identified spin in 38% of the abstracts. The most common type of spin found was type 3, occurring 40 times; the least common was type 2, which was not present in any included abstracts. We found that abstracts pertaining to pharmacologic interventions were 3.84 times more likely to contain spin than the reference group. The likelihood of an article containing spin has decreased annually (AOR: 0.91; 95% CI, 0.84-0.99). No significant correlation between funding source, other study characteristics, and the presence of spin was identified. CONCLUSIONS We have found that spin is fairly common in the abstracts of systematic reviews of melanoma treatments, but is improving.

Download Full-text

Survey of the Reporting Characteristics of Systematic Reviews in Rehabilitation

Physical Therapy ◽

10.2522/ptj.20120382 ◽

2013 ◽

Vol 93 (11) ◽

pp. 1456-1466 ◽

Cited By ~ 22

Author(s):

Silvia Gianola ◽

Monica Gasparini ◽

Michela Agostini ◽

Greta Castellini ◽

Davide Corbetta ◽

...

Keyword(s):

Systematic Reviews ◽

Controlled Trial ◽

Functional Limitations ◽

Reporting Quality ◽

Cross Sectional Study ◽

Quality Of Reporting ◽

Cross Sectional ◽

Item Data ◽

Meta Analyses

Background Systematic reviews (SRs) have become increasingly important for informing clinical practice; however, little is known about the reporting characteristics and the quality of the SRs relevant to the practice of rehabilitation health professionals. Objective The purpose of this study was to examine the reporting quality of a representative sample of published SRs on rehabilitation, focusing on the descriptive, reporting, and bias-related characteristics. Methods A cross-sectional study was conducted by searching MEDLINE for aggregative and configurative SRs indexed in 2011 that focused on rehabilitation as restorative of functional limitations. Two reviewers independently screened and selected the SRs and extracted data using a 38-item data collection form derived from Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA). The data were analyzed descriptively. Results Eighty-eight SRs published in 59 journals were sampled. The median compliance with the PRISMA items was 17 (63%) out of 27 items (interquartile ratio=13–22 [48%–82%]). Two thirds of the SRs (n=66) focused on interventions for which efficacy is best addressed through a randomized controlled trial (RCT) design, and almost all of these SRs included RCTs (63/66 [95%]). More than two thirds of the SRs assessed the quality of primary studies (74/88 [84%]). Twenty-eight reviews (28/88 [32%]) meta-analyzed the results for at least one outcome. One half of the SRs reported positive statistically significant findings (46%), whereas a detrimental result was present only in one review. Conclusions This sample of SRs in the rehabilitation field showed heterogeneous characteristics and a moderate quality of reporting. Poor control of potential source of bias might be improved if more widely agreed-upon evidence-based reporting guidelines will be actively endorsed and adhered to by authors and journals.

Download Full-text