scholarly journals Evaluating software tools to conduct systematic reviews: a feature analysis and user survey

2021 ◽  
Vol 21 (2) ◽  
pp. 124-140
Author(s):  
Marta Pellegrini ◽  
Francesco Marsili
Keyword(s):  
Systematic Review ◽  
Educational Research ◽  
Systematic Reviews ◽  
Software Tools ◽  
Feature Analysis ◽  
Synthesis Methods ◽  
Complex Process ◽  
Investigation Methods ◽  
Functional Software ◽  
Evidence Based Decision Making

Systematic reviews are research synthesis methods increasingly used in educational research to support evidence-based decision making. The conduction of a systematic review is a complex process with several phases usually supported by software tools. These tools used at the international level are not currently common in the Italian educational research. This work describes four software tools used in the international educational research and evaluates their general functionality and specific features’ usability to conduct the systematic review phases of study screening and selection. For this purpose, this study uses two methods: a feature analysis (Kitchenham et al., 1997) and an expert survey (Harrison et al., 2020). The results of both investigation methods agree to consider Covidence and Rayyan the most functional software tools in conducting SR. Among the four tools, ASReview has the greatest potential for making the process of a SR more efficient.   Una valutazione dei software per condurre revisioni sistematiche: analisi delle caratteristiche e sondaggio a esperti Le revisioni sistematiche sono metodi di sintesi di ricerca sempre più utilizzati in campo educativo al fine di supportare il processo decisionale basato su evidenze. La conduzione di una revisione sistematica, poiché complessa e su più fasi, è sovente supportata in ambito internazionale da software specifici attualmente poco diffusi nella ricerca educativa italiana. Il contributo presenta quattro software utilizzati nella ricerca educativa internazionale e valuta le funzionalità generali e l’usabilità di caratteristiche specifiche per condurre le fasi di screening e di selezione degli studi. A questo scopo lo studio impiega i metodi dell’analisi delle caratteristiche (Kitchenham et al., 1997) e del sondaggio ad esperti in revisioni sistematiche (Harrison et al., 2020) in campo educativo. I risultati di entrambi i metodi di indagine concordano nel ritenere Covidence e Rayyan gli strumenti più funzionali per condurre revisioni sistematiche, mentre ASReview risulta il software con maggiore potenzialità per rendere il processo più efficiente.

Developing, Implementing, and Applying Novel Techniques During Systematic Reviews of Primary Space Medicine Data

2021 ◽  
Vol 92 (8) ◽  
pp. 681-688
Author(s):  
Andrew Winnard ◽  
Nick Caplan ◽  
Claire Bruce-Martin ◽  
Patrick Swain ◽  
Rochelle Velho ◽  
...  
Keyword(s):  
Systematic Review ◽  
Systematic Reviews ◽  
Quantitative Methods ◽  
Primary Data ◽  
Controlled Trials ◽  
Synthesis Methods ◽  
Space Medicine ◽  
Systematic Review Methods ◽  
Meta Analyses ◽  
Primary Space

BACKGROUND: The Aerospace Medicine Systematic Review Group was set up in 2016 to facilitate high quality and transparent synthesis of primary data to enable evidence-based practice. The group identified many research methods specific to space medicine that need consideration for systematic review methods. The group has developed space medicine specific methods to address this and trialed usage of these methods across seven published systematic reviews. This paper outlines evolution of space medicine synthesis methods and discussion of their initial application.METHODS: Space medicine systematic review guidance has been developed for protocol planning, quantitative and qualitative synthesis, sourcing gray data, and assessing quality and transferability of space medicine human spaceflight simulation study environments.RESULTS: Decision algorithms for guidance and tool usage were created based on usage. Six reviews used quantitative methods in which no meta-analyses were possible due to lack of controlled trials or reporting issues. All reviews scored the quality and transferability of space simulation environments. One review was qualitative. Several research gaps were identified.CONCLUSION: Successful use of the developed methods demonstrates usability and initial validity. The current space medicine evidence base resulting in no meta-analyses being possible shows the need for standardized guidance on how to synthesize data in this field. It also provides evidence to call for increasing use of controlled trials, standardizing outcome measures, and improving minimum reporting standards. Space medicine is a unique field of medical research that requires specific systematic review methods.Winnard A, Caplan N, Bruce-Martin C, Swain P, Velho R, Meroni R, Wotring V, Damann V, Weber T, Evetts S, Laws J. Developing, implementing, and applying novel techniques during systematic reviews of primary space medicine data. Aerosp Med Hum Perform. 2021; 92(8):681688.

scholarly journals Methodology and reporting of systematic reviews and meta-analyses of observational studies in psychiatric epidemiology: Systematic review

2012 ◽  
Vol 200 (6) ◽  
pp. 446-453 ◽  
Author(s):  
Traolach S. Brugha ◽  
Ruth Matthews ◽  
Zoe Morgan ◽  
Trevor Hill ◽  
Jordi Alonso ◽  
...  
Keyword(s):  
Systematic Review ◽  
Systematic Reviews ◽  
Meta Analysis ◽  
Epidemiological Studies ◽  
Vital Role ◽  
Primary Study ◽  
Synthesis Methods ◽  
Significant Heterogeneity ◽  
Many Sources ◽  
Meta Analyses

BackgroundRelatively little is known of the use of systematic review and synthesis methods of non-randomised psychiatric epidemiological studies, which play a vital role in aetiological research, planning and policy-making.AimsTo evaluate reviews of psychiatric epidemiological studies of functional mental disorders that employed synthesis methods such as systematic review or meta-analysis, or other forms of quantitative review.MethodWe searched the literature to identify appropriate reviews published during the period 1996 to April 2009. Selected reviews were evaluated using published review guidelines.ResultsWe found 106 reviews in total, of which 38 (36%) did not mention method of data abstraction from primary studies at all. Many failed to mention study quality, publication bias, bias and confounding. In 73 studies that performed a meta-analysis, 58 (79%) tested for heterogeneity and of these, 47 found significant heterogeneity. Studies that detected heterogeneity made some allowance for this. A major obstacle facing reviewers is the wide variation between primary studies in the use of instruments to measure outcomes and in sampling methods used.ConclusionsMany deficiencies found in systematic reviews are potentially remediable, although synthesis of primary study findings in a field characterised by so many sources of heterogeneity will remain challenging.

scholarly journals Acting in the Face of Evidentiary Ambiguity, Bias, and Absence Arising from Systematic Reviews in Applied Environmental Science

2021 ◽  
Author(s):  
Trina Rytwinski ◽  
Steven J Cooke ◽  
Jessica J Taylor ◽  
Dominique Roche ◽  
Paul A Smith ◽  
...  
Keyword(s):  
Systematic Review ◽  
Decision Making ◽  
Systematic Reviews ◽  
Environmental Science ◽  
Evidence Synthesis ◽  
Empirical Studies ◽  
Synthesis Method ◽  
Evidence Base ◽  
Evidence Based ◽  
Evidence Based Decision Making

Evidence-based decision-making often depends on some form of a synthesis of previous findings. There is growing recognition that systematic reviews, which incorporate a critical appraisal of evidence, are the gold standard synthesis method in applied environmental science. Yet, on a daily basis, environmental practitioners and decision-makers are forced to act even if the evidence base to guide them is insufficient. For example, it is not uncommon for a systematic review to conclude that an evidence base is large but of low reliability. There are also instances where the evidence base is sparse (e.g., one or two empirical studies on a particular taxa or intervention), and no additional evidence arises from a systematic review. In some cases, the systematic review highlights considerable variability in the outcomes of primary studies, which in turn generates ambiguity (e.g., potentially context specific). When the environmental evidence base is ambiguous, biased, or lacking of new information, practitioners must still make management decisions. Waiting for new, higher validity research to be conducted is often unrealistic as many decisions are urgent. Here, we identify the circumstances that can lead to ambiguity, bias, and the absence of additional evidence arising from systematic reviews and provide practical guidance to resolve or handle these scenarios when encountered. Our perspective attempts to highlight that, with evidence synthesis, there may be a need to balance the spirit of evidence-based decision-making and the practical reality that management and conservation decisions and action is often time sensitive.

Hotspots in Psychology – 2020 Edition

2020 ◽  
Vol 228 (1) ◽  
pp. 1-2
Author(s):  
Michael Bošnjak ◽  
Nadine Wedderhoff
Keyword(s):  
Systematic Reviews ◽  
Research Synthesis ◽  
Synthesis Methods ◽  
Common Denominator ◽  
The Common ◽  
Meta Analyses

Abstract. This editorial gives a brief introduction to the six articles included in the fourth “Hotspots in Psychology” of the Zeitschrift für Psychologie. The format is devoted to systematic reviews and meta-analyses in research-active fields that have generated a considerable number of primary studies. The common denominator is the research synthesis nature of the included articles, and not a specific psychological topic or theme that all articles have to address. Moreover, methodological advances in research synthesis methods relevant for any subfield of psychology are being addressed. Comprehensive supplemental material to the articles can be found in PsychArchives ( https://www.psycharchives.org ).

Systematic review methodology for development: An example from microfinance

2018 ◽  
Vol 43 (1) ◽  
pp. 65-77 ◽  
Author(s):  
Carina Van Rooyen ◽  
Ruth Stewart ◽  
Thea De Wet
Keyword(s):  
Systematic Review ◽  
International Development ◽  
Health Sector ◽  
Evidence Based ◽  
Systematic Review Methodology ◽  
Development Interventions ◽  
The Us ◽  
Sub Saharan ◽  
The Impact ◽  
Evidence Based Decision Making

Big international development donors such as the UK’s Department for International Development and USAID have recently started using systematic review as a methodology to assess the effectiveness of various development interventions to help them decide what is the ‘best’ intervention to spend money on. Such an approach to evidence-based decision-making has long been practiced in the health sector in the US, UK, and elsewhere but it is relatively new in the development field. In this article we use the case of a systematic review of the impact of microfinance on the poor in sub-Saharan African to indicate how systematic review as a methodology can be used to assess the impact of specific development interventions.

The Clinical Utility and Assessment of Renal Biomarkers in Acute Kidney Injury After Abdominal Endovascular Aneurysm Repair. A Systematic Review

2020 ◽  
Vol 25 (44) ◽  
pp. 4695-4701 ◽  
Author(s):  
Georgios Karaolanis ◽  
Zachary F. Williams ◽  
Chris Bakoyiannis ◽  
Dimitrios Hadjis ◽  
Mitchell W. Cox ◽  
...  
Keyword(s):  
Systematic Review ◽  
Acute Kidney Injury ◽  
Systematic Reviews ◽  
Clinical Utility ◽  
Binding Protein ◽  
Endovascular Aneurysm Repair ◽  
Kidney Injury ◽  
Aortic Aneurysms ◽  
Renal Biomarkers ◽  
Aneurysm Repair

: The widespread adoption of endovascular aneurysm repair (EVAR) for abdominal aortic aneurysms (AAA) is due to the obvious advantages of the procedure compared to the traditional open repair. However, these advantages have to be weighed against the increased risk of renal dysfunction with EVAR. The evaluation of the perioperative renal function after EVAR has been hampered by the lack of sensitive and specific biochemical markers of acute kidney injury (AKI). The purpose of this study was to summarize all novel renal biomarkers and to evaluate their clinical utility for the assessment of the kidney function after EVAR. A systematic review of the current literature, as the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement guidelines, was performed to identify relevant studies with novel renal biomarkers and EVAR. Pubmed and Scopus databases were systemically searched. Studies reporting on thoracic endovascular aortic repair (TEVAR), case reports, case series, letters to the editor, and systematic reviews were excluded. Neutrophil-Gelatinase-Associated Lipocalin, Cystatin C, Liver-type fatty-acid-binding protein were the most common among the eligible studies while Interleukin-18, Retinol binding protein, N-acetyle-b-D-glucosaminidase and microalbumin have a sparse appearance in the literature. These biomarkers have been assessed in plasma as well as urine samples with each sample material having its own advantages and drawbacks. Which of these biomarkers has the most potential for assessing postoperative renal failure after EVAR, remains to be proved. The few studies presented in the literature show the potential clinical utility of these biomarkers, but larger studies with longer follow-up are required to determine the precise relationship between these biomarkers and postoperative acute kidney injury.

scholarly journals Appraisal of systematic reviews on interventions for postpartum depression: systematic review

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Ryan Chow ◽  
Eileen Huang ◽  
Allen Li ◽  
Sophie Li ◽  
Sarah Y. Fu ◽  
...  
Keyword(s):  
Systematic Review ◽  
Postpartum Depression ◽  
Systematic Reviews ◽  
Methodological Quality ◽  
A Priori ◽  
Behavioural Therapy ◽  
Therapeutic Interventions ◽  
Current Evidence ◽  
Cochrane Library ◽  
Measurement Tool

Abstract Background Postpartum depression (PPD) is a highly prevalent mental health problem that affects parental health with implications for child health in infancy, childhood, adolescence and beyond. The primary aim of this study was to critically appraise available systematic reviews describing interventions for PPD. The secondary aim was to evaluate the methodological quality of the included systematic reviews and their conclusions. Methods An electronic database search of MEDLINE, Embase, and the Cochrane Library from 2000 to 2020 was conducted to identify systematic reviews that examined an intervention for PPD. A Measurement Tool to Assess Systematic Reviews was utilized to independently score each included systematic review which was then critically appraised to better define the most effective therapeutic options for PPD. Results Of the 842 studies identified, 83 met the a priori criteria for inclusion. Based on the systematic reviews with the highest methodological quality, we found that use of antidepressants and telemedicine were the most effective treatments for PPD. Symptoms of PPD were also improved by traditional herbal medicine and aromatherapy. Current evidence for physical exercise and cognitive behavioural therapy in treating PPD remains equivocal. A significant, but weak relationship between AMSTAR score and journal impact factor was observed (p = 0.03, r = 0.24; 95% CI, 0.02 to 0.43) whilst no relationship was found between the number of total citations (p = 0.27, r = 0.12; 95% CI, − 0.09 to 0.34), or source of funding (p = 0.19). Conclusion Overall the systematic reviews on interventions for PPD are of low-moderate quality and are not improving over time. Antidepressants and telemedicine were the most effective therapeutic interventions for PPD treatment.

scholarly journals Determining the influence of intervention characteristics on implementation success requires reliable and valid measures: Results from a systematic review

2021 ◽  
Vol 2 ◽  
pp. 263348952199419
Author(s):  
Cara C Lewis ◽  
Kayne Mettert ◽  
Aaron R Lyon
Keyword(s):  
Systematic Review ◽  
Psychometric Properties ◽  
Systematic Reviews ◽  
Quality Measures ◽  
Measure Development ◽  
Relative Advantage ◽  
Design Quality ◽  
High Quality ◽  
Evidence Strength

Background: Despite their inclusion in Rogers’ seminal diffusion of innovations theory, few implementation studies empirically evaluate the role of intervention characteristics. Now, with growing evidence on the role of adaptation in implementation, high-quality measures of characteristics such as adaptability, trialability, and complexity are needed. Only two systematic reviews of implementation measures captured those related to the intervention or innovation and their assessment of psychometric properties was limited. This manuscript reports on the results of eight systematic reviews of measures of intervention characteristics with nuanced data regarding a broad range of psychometric properties. Methods: The systematic review proceeded in three phases. Phase I, data collection, involved search string generation, title and abstract screening, full text review, construct assignment, and citation searches. Phase II, data extraction, involved coding psychometric information. Phase III, data analysis, involved two trained specialists independently rating each measure using PAPERS (Psychometric And Pragmatic Evidence Rating Scales). Results: Searches identified 16 measures or scales: zero for intervention source, one for evidence strength and quality, nine for relative advantage, five for adaptability, six for trialability, nine for complexity, and two for design quality and packaging. Information about internal consistency and norms was available for most measures, whereas information about other psychometric properties was most often not available. Ratings for psychometric properties fell in the range of “poor” to “good.” Conclusion: The results of this review confirm that few implementation scholars are examining the role of intervention characteristics in behavioral health studies. Significant work is needed to both develop new measures (e.g., for intervention source) and build psychometric evidence for existing measures in this forgotten domain. Plain Language Summary Intervention characteristics have long been perceived as critical factors that directly influence the rate of adopting an innovation. It remains unclear the extent to which intervention characteristics including relative advantage, complexity, trialability, intervention source, design quality and packaging, evidence strength and quality, adaptability, and cost impact implementation of evidence-based practices in behavioral health settings. To unpack the differential influence of these factors, high quality measures are needed. Systematic reviews can identify measures and synthesize the data regarding their quality to identify gaps in the field and inform measure development and testing efforts. Two previous reviews identified measures of intervention characteristics, but they did not provide information about the extent of the existing evidence nor did they evaluate the host of evidence available for identified measures. This manuscript summarizes the results of nine systematic reviews (i.e., one for each of the factors listed above) for which 16 unique measures or scales were identified. The nuanced findings will help direct measure development work in this forgotten domain.

Sign in / Sign up

Export Citation Format

Share Document