scholarly journals Measuring characteristics of individuals: An updated systematic review of instruments’ psychometric properties

2021 ◽  
Vol 2 ◽  
pp. 263348952110004
Author(s):  
Cameo Stanick ◽  
Heather Halko ◽  
Kayne Mettert ◽  
Caitlin Dorsey ◽  
Joanna Moullin ◽  
...  
Keyword(s):  
Systematic Review ◽  
Psychometric Properties ◽  
Stage Of Change ◽  
Individual Characteristics ◽  
Individual Identification ◽  
Personal Attributes ◽  
Evidence Based ◽  
Individual Stage ◽  
High Quality ◽  
Knowledge And Beliefs

Background: Identification of psychometrically strong implementation measures could (1) advance researchers’ understanding of how individual characteristics impact implementation processes and outcomes, and (2) promote the success of real-world implementation efforts. The current study advances the work that our team published in 2015 by providing an updated and enhanced systematic review that identifies and evaluates the psychometric properties of implementation measures that assess individual characteristics. Methods: A full description of our systematic review methodology, which included three phases, is described in a previously published protocol paper. Phase I focused on data collection and involved search string generation, title and abstract screening, full-text review, construct assignment, and measure forward searches. During Phase II, we completed data extraction (i.e., coding psychometric information). Phase III involved data analysis, where two trained specialists independently rated each measurement tool using our psychometric rating criteria. Results: Our team identified 124 measures of individual characteristics used in mental or behavioral health research, and 123 of those measures were deemed suitable for rating using Psychometric and Pragmatic Evidence Rating Scale. We identified measures of knowledge and beliefs about the intervention ( n = 76), self-efficacy ( n = 24), individual stage of change ( n = 2), individual identification with organization ( n = 7), and other personal attributes ( n = 15). While psychometric information was unavailable and/or unreported for many measures, information about internal consistency and norms were the most commonly identified psychometric data across all individual characteristics’ constructs. Ratings for all psychometric properties predominantly ranged from “poor” to “good.” Conclusion: The majority of research that develops, uses, or examines implementation measures that evaluate individual characteristics does not include the psychometric properties of those measures. The development and use of psychometric reporting standards could advance the use of valid and reliable tools within implementation research and practice, thereby enhancing the successful implementation and sustainment of evidence-based practice in community care. Plain Language Summary: Measurement is the foundation for advancing practice in health care and other industries. In the field of implementation science, the state of measurement is only recently being targeted as an area for improvement, given that high-quality measures need to be identified and utilized in implementation work to avoid developing another research to practice gap. For the current study, we utilized the Consolidated Framework for Implementation Research to identify measures related to individual characteristics’ constructs, such as knowledge and beliefs about the intervention, self-efficacy, individual identification with the organization, individual stage of change, and other personal attributes. Our review showed that many measures exist for certain constructs (e.g., measures related to assessing providers’ attitudes and perceptions about evidence-based practice interventions), while others have very few (e.g., an individual’s stage of change). Also, we rated measures for their psychometric strength utilizing an anchored rating system and found that most measures assessing individual characteristics are in need of more research to establish their evidence of quality. It was also clear from our results that frequency of use/citations does not equate to high quality, psychometric strength. Ultimately, the state of the literature has demonstrated that assessing individual characteristics of implementation stakeholders is an area of strong interest in implementation work. It will be important for future research to focus on clearly delineating the psychometric properties of existing measures for saturated constructs, while for the others the emphasis should be on developing new, high-quality measures and make these available to stakeholders.

scholarly journals Determining the influence of intervention characteristics on implementation success requires reliable and valid measures: Results from a systematic review

2021 ◽  
Vol 2 ◽  
pp. 263348952199419
Author(s):  
Cara C Lewis ◽  
Kayne Mettert ◽  
Aaron R Lyon
Keyword(s):  
Systematic Review ◽  
Psychometric Properties ◽  
Systematic Reviews ◽  
Quality Measures ◽  
Measure Development ◽  
Relative Advantage ◽  
Design Quality ◽  
High Quality ◽  
Evidence Strength

Background: Despite their inclusion in Rogers’ seminal diffusion of innovations theory, few implementation studies empirically evaluate the role of intervention characteristics. Now, with growing evidence on the role of adaptation in implementation, high-quality measures of characteristics such as adaptability, trialability, and complexity are needed. Only two systematic reviews of implementation measures captured those related to the intervention or innovation and their assessment of psychometric properties was limited. This manuscript reports on the results of eight systematic reviews of measures of intervention characteristics with nuanced data regarding a broad range of psychometric properties. Methods: The systematic review proceeded in three phases. Phase I, data collection, involved search string generation, title and abstract screening, full text review, construct assignment, and citation searches. Phase II, data extraction, involved coding psychometric information. Phase III, data analysis, involved two trained specialists independently rating each measure using PAPERS (Psychometric And Pragmatic Evidence Rating Scales). Results: Searches identified 16 measures or scales: zero for intervention source, one for evidence strength and quality, nine for relative advantage, five for adaptability, six for trialability, nine for complexity, and two for design quality and packaging. Information about internal consistency and norms was available for most measures, whereas information about other psychometric properties was most often not available. Ratings for psychometric properties fell in the range of “poor” to “good.” Conclusion: The results of this review confirm that few implementation scholars are examining the role of intervention characteristics in behavioral health studies. Significant work is needed to both develop new measures (e.g., for intervention source) and build psychometric evidence for existing measures in this forgotten domain. Plain Language Summary Intervention characteristics have long been perceived as critical factors that directly influence the rate of adopting an innovation. It remains unclear the extent to which intervention characteristics including relative advantage, complexity, trialability, intervention source, design quality and packaging, evidence strength and quality, adaptability, and cost impact implementation of evidence-based practices in behavioral health settings. To unpack the differential influence of these factors, high quality measures are needed. Systematic reviews can identify measures and synthesize the data regarding their quality to identify gaps in the field and inform measure development and testing efforts. Two previous reviews identified measures of intervention characteristics, but they did not provide information about the extent of the existing evidence nor did they evaluate the host of evidence available for identified measures. This manuscript summarizes the results of nine systematic reviews (i.e., one for each of the factors listed above) for which 16 unique measures or scales were identified. The nuanced findings will help direct measure development work in this forgotten domain.

scholarly journals Measures of Facilitator Competent Adherence Used in Parenting Programs and Their Psychometric Properties: A Systematic Review

2021 ◽  
Author(s):  
Mackenzie Martin ◽  
Bridget Steele ◽  
Jamie M. Lachman ◽  
Frances Gardner
Keyword(s):  
Systematic Review ◽  
Psychometric Properties ◽  
Scale Up ◽  
Implementation Fidelity ◽  
Parenting Programs ◽  
High Quality ◽  
Observational Measures ◽  
Intervention Science ◽  
Cosmin Checklist ◽  
Registration Number

AbstractImplementation fidelity is a critical component of intervention science, which aims to understand how interventions unfold in practice to improve outcomes. A key element of fidelity is facilitator competent adherence—the extent to which a program is delivered as prescribed with the specified level of quality. We conducted a two-part systematic review examining these aspects in parenting programs aiming to reduce child behavior problems and maltreatment. Part One reviews measures of facilitator competent adherence and Part Two examines the psychometric properties of the observational measures found. Searches identified 9153 articles from electronic databases, citation tracking, and expert input. After screening using pre-specified criteria, 156 (Part One) and 41 (Part Two) articles remained. In Part One, measure, facilitator, and intervention characteristics were extracted and synthesized from 65 measures. Most measures were observational, used by facilitators and researchers, and employed Likert-scale ratings. In Part Two, evidence on the reliability (internal consistency, inter-rater, intra-rater, test–retest) and validity (content, construct, convergent/divergent, criterion) of 30 observational measures identified from Part One was synthesized and evaluated. An adapted COSMIN checklist was used to assess study and measure quality. We found most studies to be of reasonably high quality. This is the first review to summarize and critically appraise measures of facilitator competent adherence used in the parenting program literature and establish their psychometric properties. The findings underscore the need to advance research on measures of facilitator competent adherence; reliable, valid, and high-quality implementation measures allow for evidence-based decisions regarding the delivery and scale-up of parenting programs. PROSPERO Registration Number: CRD42020167872.

scholarly journals Effectiveness of Auricular Acupressure for Post-Stroke Constipation: A Systematic Review and Meta-Analysis

2021 ◽  
Vol 7 (6) ◽  
pp. 1-8
Author(s):  
Irene J Su ◽  
Keyword(s):  
Systematic Review ◽  
Chinese Medicine ◽  
Traditional Chinese Medicine ◽  
Evidence Based Practice ◽  
Meta Analysis ◽  
Evidence Based ◽  
High Quality ◽  
Post Stroke ◽  
Auricular Acupressure

The preliminary findings of this study concluded that auricular acupressure was an effective and safe measure of intervention for the management of post-stroke constipation. We are hoping to see more high-quality studies to validate this result to disseminate the evidence-based practice of Traditional Chinese medicine (TCM).

scholarly journals A systematic review of the psychometric properties, usability and clinical impacts of mobile mood-monitoring applications in young people

2017 ◽  
Vol 48 (2) ◽  
pp. 208-228 ◽  
Author(s):  
M. Dubad ◽  
C. Winsper ◽  
C. Meyer ◽  
M. Livanou ◽  
S. Marwaha
Keyword(s):  
Mental Health ◽  
Systematic Review ◽  
Young People ◽  
Psychometric Properties ◽  
Self Regulation ◽  
Emotional Awareness ◽  
Individual Characteristics ◽  
Depression Symptoms ◽  
Monitoring Applications ◽  
Negative Impacts

BackgroundMobile mood-monitoring applications are increasingly used by mental health providers, widely advocated within research, and a potentially effective method to engage young people. However, little is known about their efficacy and usability in young populations.MethodA systematic review addressing three research questions focused on young people: (1) what are the psychometric properties of mobile mood-monitoring applications; (2) what is their usability; and (3) what are their positive and negative clinical impacts? Findings were synthesised narratively, study quality assessed and compared with evidence from adult studies.ResultsWe reviewed 25 articles. Studies on the psychometric properties of mobile mood-monitoring applications were sparse, but indicate questionable to excellent internal consistency, moderate concurrent validity and good usability. Participation rates ranged from 30% to 99% across studies, and appeared to be affected by methodological factors (e.g. payments) and individual characteristics (e.g. IQ score). Mobile mood-monitoring applications are positively perceived by youth, may reduce depressive symptoms by increasing emotional awareness, and could aid in the detection of mental health and substance use problems. There was very limited evidence on potential negative impacts.ConclusionsEvidence for the use of mood-monitoring applications in youth is promising but limited due to a lack of high-quality studies. Future work should explicate the effects of mobile mood-monitoring applications on effective self-regulation, clinical outcomes across disorders and young people's engagement with mental health services. Potential negative impacts in this population should also be investigated, as the adult literature suggests that application use could potentially increase negativity and depression symptoms.

How to write a systematic review: Tips and tricks for surgeons in training

2016 ◽  
Vol 10 (2) ◽  
pp. 148-153
Author(s):  
Patrick Jones ◽  
Helen Ewan ◽  
Timothy Lane ◽  
Jim Adshead ◽  
Nikhil Vasdev ◽  
...  
Keyword(s):  
Systematic Review ◽  
Patient Care ◽  
Systematic Reviews ◽  
Critical Appraisal ◽  
Evidence Based ◽  
Tips And Tricks ◽  
High Quality

Systematic reviews provide high-quality critical appraisal and evidence-based summaries on a topic. They represent a key resource for time-pressured clinicians as they strive to deliver better patient care. Robust methodology and adhering to rigorous standards forms the foundation of this type of article. As such, writing a systematic review can prove a great challenge. This article aims to provide an overview of the methodology as well as certain tips and tricks which will help the surgeon when taking on such a project.

scholarly journals Patient Engagement and its Evaluation Tools – Current Challenges and Future Directions Comment on "Metrics and Evaluation Tools for Patient Engagement in Healthcare Organization- and System-Level Decision-Making: A Systematic Review"

2019 ◽  
Vol 8 (6) ◽  
pp. 378-380 ◽  
Author(s):  
Samira Abbasgholizadeh Rahimi ◽  
Hervé Tchala Vignon Zomahoun ◽  
France Légaré
Keyword(s):  
Systematic Review ◽  
Decision Making ◽  
Psychometric Properties ◽  
Patient Engagement ◽  
Delivery Systems ◽  
Healthcare Organization ◽  
System Level ◽  
Evaluation Tools ◽  
High Quality ◽  
Future Directions

Considering the growing recognition of the importance of patient engagement in healthcare decisions, research and delivery systems, it is important to ensure high quality and efficient patient engagement evaluation tools. In this commentary, we will first highlight the definition and importance of patient engagement. Then we discuss the psychometric properties of the patient engagement evaluation tools identified in a recent review on patient engagement in healthcare organization- and system-level decision-making. Lastly, we suggest future directions for patient engagement and its evaluation tools.

Evidence-Based Practice & Practice-Based Evidence Applied to Adult, Medical Speech-Language Pathology

2013 ◽  
Vol 18 (1) ◽  
pp. 14-26 ◽  
Author(s):  
Rik Lemoncello ◽  
Bryan Ness
Keyword(s):  
Evidence Based Practice ◽  
Single Case ◽  
Quality Data ◽  
Evidence Based ◽  
Speech Language Pathology ◽  
Clinical Practices ◽  
High Quality ◽  
Randomized Controlled Clinical Trials ◽  
Relative Paucity ◽  
Language Pathology

In this paper, we review concepts of evidence-based practice (EBP), and provide a discussion of the current limitations of EBP in terms of a relative paucity of efficacy evidence and the limitations of applying findings from randomized controlled clinical trials to individual clinical decisions. We will offer a complementary model of practice-based evidence (PBE) to encourage clinical scientists to design, implement, and evaluate our own clinical practices with high-quality evidence. We will describe two models for conducting PBE: the multiple baseline single-case experimental design and a clinical case study enhanced with generalization and control data probes. Gathering, analyzing, and sharing high-quality data can offer additional support through PBE to support EBP in speech-language pathology. It is our hope that these EBP and PBE strategies will empower clinical scientists to persevere in the quest for best practices.

Systematic review methodology for development: An example from microfinance

2018 ◽  
Vol 43 (1) ◽  
pp. 65-77 ◽  
Author(s):  
Carina Van Rooyen ◽  
Ruth Stewart ◽  
Thea De Wet
Keyword(s):  
Systematic Review ◽  
International Development ◽  
Health Sector ◽  
Evidence Based ◽  
Systematic Review Methodology ◽  
Development Interventions ◽  
The Us ◽  
Sub Saharan ◽  
The Impact ◽  
Evidence Based Decision Making

Big international development donors such as the UK’s Department for International Development and USAID have recently started using systematic review as a methodology to assess the effectiveness of various development interventions to help them decide what is the ‘best’ intervention to spend money on. Such an approach to evidence-based decision-making has long been practiced in the health sector in the US, UK, and elsewhere but it is relatively new in the development field. In this article we use the case of a systematic review of the impact of microfinance on the poor in sub-Saharan African to indicate how systematic review as a methodology can be used to assess the impact of specific development interventions.

Sign in / Sign up

Export Citation Format

Share Document