scholarly journals A systematic review of measures of implementation players and processes: Summarizing the dearth of psychometric evidence

2021 ◽  
Vol 2 ◽  
pp. 263348952110024
Author(s):  
Caitlin N Dorsey ◽  
Kayne D Mettert ◽  
Ajeng J Puspitasari ◽  
Laura J Damschroder ◽  
Cara C Lewis
Keyword(s):  
Psychometric Properties ◽  
Behavioral Health ◽  
Discriminant Validity ◽  
Rating Scale ◽  
Structural Characteristics ◽  
Opinion Leaders ◽  
Phase Iii ◽  
Successful Implementation ◽  
Plain Language

Background: Measurement is a critical component for any field. Systematic reviews are a way to locate measures and uncover gaps in current measurement practices. The present study identified measures used in behavioral health settings that assessed all constructs within the Process domain and two constructs from the Inner setting domain as defined by the Consolidated Framework for Implementation Research (CFIR). While previous conceptual work has established the importance social networks and key stakeholders play throughout the implementation process, measurement studies have not focused on investigating the quality of how these activities are being carried out. Methods: The review occurred in three phases: Phase I, data collection included (1) search string generation, (2) title and abstract screening, (3) full text review, (4) mapping to CFIR-constructs, and (5) “cited-by” searches. Phase II, data extraction, consisted of coding information relevant to the nine psychometric properties included in the Psychometric And Pragmatic Rating Scale (PAPERS). In Phase III, data analysis was completed. Results: Measures were identified in only seven constructs: Structural characteristics ( n = 13), Networks and communication ( n = 29), Engaging ( n = 1), Opinion leaders ( n = 5), Champions ( n = 5), Planning ( n = 5), and Reflecting and evaluating ( n = 5). No quantitative assessment measures of Formally appointed implementation leaders, External change agents, or Executing were identified. Internal consistency and norms were reported on most often, whereas no studies reported on discriminant validity or responsiveness. Not one measure in the sample reported all nine psychometric properties evaluated by the PAPERS. Scores in the identified sample of measures ranged from “-2” to “10” out of a total of “36.” Conclusions: Overall measures demonstrated minimal to adequate evidence and available psychometric information was limited. The majority were study specific, limiting their generalizability. Future work should focus on more rigorous measure development and testing of currently existing measures, while moving away from creating new, single use measures. Plain Language Summary: How we measure the processes and players involved for implementing evidence-based interventions is crucial to understanding what factors are helping or hurting the intervention’s use in practice and how to take the intervention to scale. Unfortunately, measures of these factors—stakeholders, their networks and communication, and their implementation activities—have received little attention. This study sought to identify and evaluate the quality of these types of measures. Our review focused on collecting measures used for identifying influential staff members, known as opinion leaders and champions, and investigating how they plan, execute, engage, and evaluate the hard work of implementation. Upon identifying these measures, we collected all published information about their uses to evaluate the quality of their evidence with respect to their ability to produce consistent results across items within each use (i.e., reliable) and if they assess what they are intending to measure (i.e., valid). Our searches located over 40 measures deployed in behavioral health settings for evaluation. We observed a dearth of evidence for reliability and validity and when evidence existed the quality was low. These findings tell us that more measurement work is needed to better understand how to optimize players and processes for the purposes of successful implementation.

scholarly journals Investigation of the Statistical Anxiety Rating Scale Psychometric Properties with a Sample of Greek Students

2021 ◽  
Vol 10 (2) ◽  
pp. 116
Author(s):  
Konstantinos Lavidas ◽  
Dionysios Manesis ◽  
Vasilios Gialamas
Keyword(s):  
Psychometric Properties ◽  
Discriminant Validity ◽  
Rating Scale ◽  
Factor Model ◽  
Tertiary Education ◽  
Statistics Anxiety ◽  
Education Students ◽  
Confirmatory Factor Analyses ◽  
Convergent And Discriminant Validity ◽  
Confirmatory Factor

The purpose of this study was to adapt the Statistics Anxiety Rating Scale (STARS) for a Greek student population. The STARS was administered to 890 Tertiary Education students in two Greek universities. It was performed a cross-validation study to examine the factorial structure and the psychometric properties with a series of confirmatory factor analyses. Results revealed a correlated six first-order factor model which provided the best fit to the data compared to a six-factor model with one superordinate factor. All six factors of the Greek version of the STARS presented convergent and discriminant validity and were internally consistent. Implications and limitations are discussed.

scholarly journals Propiedades psicométricas de la escala de valoración de organizaciones deportivas EPOD (Psychometric properties of the rating scale sports organizations EPOD)

Retos ◽  
2015 ◽  
pp. 83-87
Author(s):  
Alberto Nuviala Nuviala ◽  
Javier Antonio Tamayo Fajardo ◽  
Román Nuviala Nuviala ◽  
José Antonio González Jurado ◽  
Antonio Fernández Martínez
Keyword(s):  
Psychometric Properties ◽  
User Satisfaction ◽  
Rating Scale ◽  
High Reliability ◽  
Perceived Quality ◽  
Sport Organizations ◽  
Four Dimensions ◽  
Sports Organizations ◽  
Internal And External Validity

La comprensión de la satisfacción de los usuarios y consumidores de servicios es una necesidad para los responsables de las mismas. Existe una diversidad de instrumentos que valoran la calidad percibida pero son criticados por diversos motivos: conceptuales, procedimentales, falta de especificidad… EPOD es una escala de 29 ítems integrados en cuatro dimensiones, de aplicación práctica y directa que había sido validada con anterioridad. Sin embargo no se había constatado las propiedades psicométricas del instrumento. Por ello, el objetivo de este trabajo es determinar las mismas. La muestra estuvo constituida por 463 usuarios de servicios deportivos a los que se les suministró el instrumento. Los resultados muestran una buena validez interna y externa, al tiempo que posee una alta fiabilidad. Por ello, EPOD se convierte en una herramienta capaz de evaluar la calidad percibida del servicio que prestan las organizaciones deportivas de forma válida, fiable, sencilla y breve, contemplando las diferentes dimensiones que conforman la prestación de los servicios deportivos.Palabra clave: escala, validación, fiabilidad, organizaciones deportivas, calidad.Abstract: The understanding of user satisfaction and consumers of services is a necessity for those responsible for them. There’s a variety of instruments that assess the perceived quality but are criticized for several reasons: concepts, procedures, lack of specificity... EPOD is a scale with 29 items integrated in four dimensions, direct practical application which had been previously validated. However, nobody had checked the psychometric properties of the instrument. Therefore, the aim of this study is to determine them. The sample was 463 users of sports services who were given the instrument. The results show good internal and external validity, while it has a high reliability. Therefore, EPOD becomes a tool to assess the perceived quality of service provided by the sports organizations in a valid, reliable, simple and brief way, considering the different dimensions that make up the provision of sports services.Key words: scale, validity, reliability, sport organizations, quality.

scholarly journals The Relationship Satisfaction scale – Psychometric properties

2014 ◽  
Vol 24 (1-2) ◽  
Author(s):  
Espen Røysamb ◽  
Joar Vittersø ◽  
Kristian Tambs
Keyword(s):  
Relationship Satisfaction ◽  
Psychometric Properties ◽  
Discriminant Validity ◽  
Satisfaction With Life ◽  
Satisfaction With Life Scale ◽  
Short Version ◽  
Relationship Dissolution ◽  
Big Five Personality ◽  
Convergent And Discriminant Validity

The aim of this study was to establish the psychometric properties of the new Relationship Satisfaction (RS) scale. Two population based samples were used: The Norwegian Mother and Child Cohort Study (MoBa, N=117,178) and The Quality of Life study (N=347). Convergent and discriminant validity was investigated in relation to the Quality of Marriage Index (QMI), the Satisfaction With Life Scale (SWLS), Relationship Satisfaction of partner, Big Five personality traits (IPIP50) and future relationship dissolution. The full scale with ten items (RS10) and a short version with five items (RS5) showed good psychometric properties. The scale has high internal and test-retest reliability and high structural, convergent, and discriminant validity. Measurement invariance across gender was established. Additionally, predictive validity was evidenced by prediction of future relationship dissolution. We conclude that the RS scale is highly useful as a generic measure of global relationship satisfaction.

Translation and validation of the stroke impact scale 3.0 into urdu for Pakistan

2021 ◽  
pp. 1-12
Author(s):  
Umair Ahmed ◽  
Hossein Karimi ◽  
Syed Amir Gilani ◽  
Ashfaq Ahmad
Keyword(s):  
Psychometric Properties ◽  
Discriminant Validity ◽  
Hand Function ◽  
Minimal Detectable Change ◽  
Stroke Impact Scale ◽  
Impact Scale ◽  
Assess Quality ◽  
Test Retest Reliability ◽  
Translation And Validation

BACKGROUND: The Stroke Impact Scale version 3.0 (SIS 3.0) is a self-reported outcome measure designed to assess quality of life (QoL) following a stroke. Although the psychometric properties of the SIS 3.0 are identified as superior to the generic QoL scales, it has not been translated and tested in Pakistan. OBJECTIVE: To validate the Urdu version of the SIS 3.0 (USIS 3.0) for Pakistan. METHODS: A prospective cohort of 116 patients with mild to moderate stroke reported their recovery using the USIS 3.0. The patients were concurrently assessed on the established tools to assess the validity and were re-evaluated to determine the test-retest reliability, precision, minimal detectable change (MDC), and minimal clinically important difference (MCID). RESULTS: The reliability and internal consistency of USIS were satisfactory except for the emotion domain. The correlations of USIS with the established tools were strong. The discriminant validity was also significant across the levels of the modified Rankin scale (MRS). Only hand function and communication domains exhibited significant floor and ceiling effects, respectively. Regarding weighted K, values ranged from 0.53 to 0.88. CONCLUSIONS: The USIS 3.0 has satisfactory psychometric properties and can be used in clinical and research settings for stroke survivors.

scholarly journals Measuring implementation outcomes: An updated systematic review of measures’ psychometric properties

2020 ◽  
Vol 1 ◽  
pp. 263348952093664 ◽  
Author(s):  
Kayne Mettert ◽  
Cara Lewis ◽  
Caitlin Dorsey ◽  
Heather Halko ◽  
Bryan Weiner
Keyword(s):  
Systematic Review ◽  
Health Care ◽  
Psychometric Properties ◽  
Behavioral Health ◽  
Internal Consistency ◽  
Behavioral Health Care ◽  
Research And Practice ◽  
Implementation Outcomes ◽  
New Treatment

Background: Systematic reviews of measures can facilitate advances in implementation research and practice by locating reliable and valid measures and highlighting measurement gaps. Our team completed a systematic review of implementation outcome measures published in 2015 that indicated a severe measurement gap in the field. Now, we offer an update with this enhanced systematic review to identify and evaluate the psychometric properties of measures of eight implementation outcomes used in behavioral health care. Methods: The systematic review methodology is described in detail in a previously published protocol paper and summarized here. The review proceeded in three phases. Phase I, data collection, involved search string generation, title and abstract screening, full text review, construct assignment, and measure forward searches. Phase II, data extraction, involved coding psychometric information. Phase III, data analysis, involved two trained specialists independently rating each measure using PAPERS (Psychometric And Pragmatic Evidence Rating Scales). Results: Searches identified 150 outcomes measures of which 48 were deemed unsuitable for rating and thus excluded, leaving 102 measures for review. We identified measures of acceptability ( N = 32), adoption ( N = 26), appropriateness ( N = 6), cost ( N = 31), feasibility ( N = 18), fidelity ( N = 18), penetration ( N = 23), and sustainability ( N = 14). Information about internal consistency and norms were available for most measures (59%). Information about other psychometric properties was often not available. Ratings for internal consistency and norms ranged from “adequate” to “excellent.” Ratings for other psychometric properties ranged mostly from “poor” to “good.” Conclusion: While measures of implementation outcomes used in behavioral health care (including mental health, substance use, and other addictive behaviors) are unevenly distributed and exhibit mostly unknown psychometric quality, the data reported in this article show an overall improvement in availability of psychometric information. This review identified a few promising measures, but targeted efforts are needed to systematically develop and test measures that are useful for both research and practice. Plain language abstract: When implementing an evidence-based treatment into practice, it is important to assess several outcomes to gauge how effectively it is being implemented. Outcomes such as acceptability, feasibility, and appropriateness may offer insight into why providers do not adopt a new treatment. Similarly, outcomes such as fidelity and penetration may provide important context for why a new treatment did not achieve desired effects. It is important that methods to measure these outcomes are accurate and consistent. Without accurate and consistent measurement, high-quality evaluations cannot be conducted. This systematic review of published studies sought to identify questionnaires (referred to as measures) that ask staff at various levels (e.g., providers, supervisors) questions related to implementation outcomes, and to evaluate the quality of these measures. We identified 150 measures and rated the quality of their evidence with the goal of recommending the best measures for future use. Our findings suggest that a great deal of work is needed to generate evidence for existing measures or build new measures to achieve confidence in our implementation evaluations.

Selecting the Best Instrument to Measure Family Perceptions of End-of-Life Communication in Intensive Care Units

2019 ◽  
Vol 37 (2) ◽  
pp. 154-160
Author(s):  
Chiahui Chen ◽  
Scott T. Meier
Keyword(s):  
Intensive Care ◽  
Psychometric Properties ◽  
End Of Life ◽  
Convergent Validity ◽  
Discriminant Validity ◽  
Outcome Instrument ◽  
Clinician Communication ◽  
Family Perceptions ◽  
Eol Care

Background: Good communication with the family is a clinical imperative for high quality end-of-life (EOL) care in intensive care unit (ICU). Many interventions aim to improve EOL communication, and the choice of an outcome instrument has important implications for evaluating interventions. The purpose of this project is to search and review available instruments’ psychometric properties and determine which best measures family–clinician communication in the ICU. Method: A stepwise method was used by searching 2 databases (PsycInfo and Web of Science) to identify instruments and articles that provide information about scale psychometric properties. Instruments: Three instruments were identified, including Family Inpatient Communication Survey, Family Perception of Physician–Family Caregiver Communication, and Quality of Communication (QOC). Results: Reliability estimates were high (≥ 0.79) in all 3 instruments. The QOC’s convergent validity estimates exceed its discriminant validity values, and the QOC is an intervention-sensitive measure used to examine families’ treatment response in randomized control trials. Conclusion: Quality of Communication is the most suitable instrument to measure family’s perceptions of EOL communication in the ICU. Quality of Communication scores provide a deeper understanding of family–clinician communication and data about how to improve EOL care in ICUs.

scholarly journals Psychometric Properties of the Croatian Version of the 25-item National Eye Institute Visual Function Questionnaire (NEI VFQ-25)

2021 ◽  
Author(s):  
Dina Lešin Gaćina ◽  
Bernarda Škegro ◽  
Sonja Jandroković ◽  
Ivan Škegro ◽  
Iva Bešlić ◽  
...  
Keyword(s):  
Quality Of Life ◽  
Visual Impairment ◽  
Psychometric Properties ◽  
Visual Function ◽  
Discriminant Validity ◽  
Ophthalmological Examination ◽  
Sf 36 ◽  
Related Quality ◽  
Visual Function Questionnaire

Abstract Purpose: The purpose of study was to translate, adapt and validate the Croatian version of the 25-item National Eye Institute Visual Function Questionnaire (NEI VFQ-25) in participants with visual impairment. This study also aims at evaluating the relationship between visual impairment and health-related quality of life (HRQoL).Methods: The prospective observational study was conducted at the University Hospital Centre Zagreb, Department of Ophthalmology. The sample consisted of 175 patients with four chronic ocular diseases: cataract, glaucoma, diabetic retinopathy and age-related macular degeneration (ARMD). The translation of the NEI VFQ-25 to Croatian was conducted following the standardized procedure. All participants underwent an ophthalmological examination and completed the NEI VFQ-25 and the Medical Outcomes Study Short Form-36 Questionnaire (SF-36). In order to assess the psychometric properties of the NEI VFQ-25 we calculated Cronbach's α coefficient, intraclass correlation coefficient (ICC), convergent and discriminant validity, as well as criterion and concurrent validity.Results: Results show high internal consistency (Cronbach α range 0.739-0.932) and high test-retest reliability (ICC 0.876-0.975) for all subscales. None of the items had failed either convergent or discriminant validity. Moderate to high Spearman's rho coefficients of correlations were found between best corrected visual acuity and 8 subscales in the NEI VFQ-25 (0.430<ρ<0.631). Moderate correlations were found between comparable domains in the NEI VFQ-25 and in the SF-36 questionnaire (p<0.01).Conclusion: The Croatian version of the NEI VFQ-25 has very good psychometric properties and can be a useful instrument for assessing vision-related quality of life in Croatian population with chronic ophthalmic diseases. The trial registration number: DRKS-ID DRKS00016751Date of registration in DRKS: 2019/02/15

The Psychometric Properties of the Scale of Retired Spousal Intrusion

2017 ◽  
Vol 33 (6) ◽  
pp. 429-435
Author(s):  
Bahadir Bozoglan
Keyword(s):  
Quality Of Life ◽  
Psychometric Properties ◽  
Discriminant Validity ◽  
Factor Model ◽  
Dyadic Adjustment ◽  
Reliable Measure ◽  
Dyadic Adjustment Scale ◽  
Confirmatory Factor ◽  
Calibration Study

Abstract. This study examined the psychometric properties of Retired Spousal Intrusion Scale (SRSI). The calibration study was conducted with a sample of women (n = 687) whose husbands were retired, resulting in a one factor scale consisting of six items. Validation study, discriminant validity and test-retest were conducted in a second sample of women (n = 756) whose husbands were retired. The results of the confirmatory factor analysis revealed that the unidimensional factor model had adequate fit with values x2/df = 4.61, GFI = .98, AGFI = .96, CFI = .96, RMSEA = .06, SRMR = .05. In addition, the correlation of the SRSI with conceptually unrelated measures (dyadic adjustment scale and quality of life) supported the discriminant validity of the SRSI. Finally, analyses showed that SRSI is a valid and reliable measure to assess retired husbands’ intrusion on their wives.

scholarly journals A systematic review of the validity and reliability of apathy scales in neurodegenerative conditions

2014 ◽  
Vol 27 (6) ◽  
pp. 903-923 ◽  
Author(s):  
Ratko Radakovic ◽  
Catherine Harley ◽  
Sharon Abrahams ◽  
John M. Starr
Keyword(s):  
Systematic Review ◽  
Neurodegenerative Diseases ◽  
Neurodegenerative Disease ◽  
Psychometric Properties ◽  
Validation Studies ◽  
Methodological Quality ◽  
Rating Scale ◽  
Validity And Reliability ◽  
Prevalent Symptom

ABSTRACTBackground:There are several scales used to detect apathy in disease populations. Since apathy is a prevalent symptom in many neurodegenerative diseases, this is an especially important context in which to identify and compare scales.Aims:To provide an overview of apathy scales validated in generic and specific neurodegenerative disease populations, compare validation studies’ methodological quality and the psychometric properties of the validated apathy scales.Methods:A systematic review of literature was conducted of articles published between 1980 and 2013. The final articles selected for review were rated on methodological quality and the psychometric properties of the scales used were interpreted.Results:Sixteen articles validating apathy scales were included in the review, five in a generic neurodegenerative sample and eleven in specific neurodegenerative samples. The methodological quality of specific studies varied from poor to excellent. The highest quality, which had psychometrically favorable scales, were the dementia apathy interview and rating (DAIR) and the apathy evaluation scale-clinical version (AES-C) in Alzheimer's disease and the Lille apathy rating scale (LARS) in Parkinson's disease. Generic neurodegenerative disease validation studies were of average methodological quality and yielded inconsistent psychometric properties.Conclusions:Several instruments can be recommended for use in some specific neurodegenerative diseases. Other instruments should either be validated or developed to assess apathy in more generic populations.

Sign in / Sign up

Export Citation Format

Share Document