A Reliability Generalization Meta-analysis of the Padua Inventory-Revised (PI-R)

Rosa María Núñez-Núñez; María Rubio-Aparicio; Fulgencio Marín-Martínez; Julio Sánchez-Meca; José Antonio López-Pina; José Antonio López-López

doi:10.1016/j.ijchp.2021.100277

The Fear of COVID-19 Scale: A Reliability Generalization Meta-Analysis

Assessment ◽

10.1177/1073191121994164 ◽

2021 ◽

pp. 107319112199416

Author(s):

Desirée Blázquez-Rincón ◽

Juan I. Durán ◽

Juan Botella

Keyword(s):

Predictive Model ◽

Meta Analysis ◽

Classical Test Theory ◽

Measurement Model ◽

Rasch Measurement ◽

Test Theory ◽

Reliability Estimate ◽

Reliability Generalization ◽

Reliability Estimates ◽

Total Variability

A reliability generalization meta-analysis was carried out to estimate the average reliability of the seven-item, 5-point Likert-type Fear of COVID-19 Scale (FCV-19S), one of the most widespread scales developed around the COVID-19 pandemic. Different reliability coefficients from classical test theory and the Rasch Measurement Model were meta-analyzed, heterogeneity among the most reported reliability estimates was examined by searching for moderators, and a predictive model to estimate the expected reliability was proposed. At least one reliability estimate was available for a total of 44 independent samples out of 42 studies, being that Cronbach’s alpha was most frequently reported. The coefficients exhibited pooled estimates ranging from .85 to .90. The moderator analyses led to a predictive model in which the standard deviation of scores explained 36.7% of the total variability among alpha coefficients. The FCV-19S has been shown to be consistently reliable regardless of the moderator variables examined.

Download Full-text

A Test-Retest Reliability Generalization Meta-Analysis of Judgments Via the Policy-Capturing Technique

Organizational Research Methods ◽

10.1177/10944281211011529 ◽

2021 ◽

pp. 109442812110115

Author(s):

Ze Zhu ◽

Alan J. Tomassetti ◽

Reeshad S. Dalal ◽

Shannon W. Schrader ◽

Kevin Loo ◽

...

Keyword(s):

Best Practice ◽

Meta Analysis ◽

Temporal Stability ◽

Future Research ◽

Policy Capturing ◽

Reliability Estimate ◽

Reliability Generalization ◽

Retest Reliability ◽

Reliability Estimates ◽

Test Retest Reliability

Policy capturing is a widely used technique, but the temporal stability of policy-capturing judgments has long been a cause for concern. This article emphasizes the importance of reporting reliability, and in particular test-retest reliability, estimates in policy-capturing studies. We found that only 164 of 955 policy-capturing studies (i.e., 17.17%) reported a test-retest reliability estimate. We then conducted a reliability generalization meta-analysis on policy-capturing studies that did report test-retest reliability estimates—and we obtained an average reliability estimate of .78. We additionally examined 16 potential methodological and substantive antecedents to test-retest reliability (equivalent to moderators in validity generalization studies). We found that test-retest reliability was robust to variation in 14 of the 16 factors examined but that reliability was higher in paper-and-pencil studies than in web-based studies and was higher for behavioral intention judgments than for other (e.g., attitudinal and perceptual) judgments. We provide an agenda for future research. Finally, we provide several best-practice recommendations for researchers (and journal reviewers) with regard to (a) reporting test-retest reliability, (b) designing policy-capturing studies for appropriate reportage, and (c) properly interpreting test-retest reliability in policy-capturing studies.

Download Full-text

A Reliability Generalization Meta-Analysis of Runco Ideational Behavior Scale

Creativity Research Journal ◽

10.1080/10400419.2021.1960719 ◽

2021 ◽

pp. 1-17

Author(s):

Sedat Sen

Keyword(s):

Meta Analysis ◽

Reliability Generalization ◽

Behavior Scale

Download Full-text

A Reliability Generalization Meta-Analysis of Coefficient Alpha for the Maslach Burnout Inventory

Educational and Psychological Measurement ◽

10.1177/0013164410391579 ◽

2011 ◽

Vol 71 (1) ◽

pp. 231-244 ◽

Cited By ~ 54

Author(s):

Denna L. Wheeler ◽

Matt Vassar ◽

Jody A. Worley ◽

Laura L. B. Barnes

Keyword(s):

Meta Analysis ◽

Maslach Burnout Inventory ◽

Coefficient Alpha ◽

Reliability Generalization

Download Full-text

The Padua Inventory–Washington State University Revision of Obsessions and Compulsions: A Reliability Generalization Meta-Analysis

Journal of Personality Assessment ◽

10.1080/00223891.2018.1483378 ◽

2018 ◽

Vol 102 (1) ◽

pp. 113-123 ◽

Cited By ~ 2

Author(s):

María Rubio-Aparicio ◽

Rosa M. Núñez-Núñez ◽

Julio Sánchez-Meca ◽

José Antonio López-Pina ◽

Fulgencio Marín-Martínez ◽

...

Keyword(s):

Meta Analysis ◽

Washington State ◽

State University ◽

Reliability Generalization ◽

Washington State University

Download Full-text

Reliability Generalization as a Seal of Quality of Substantive Meta-Analyses: The Case of the VIA Inventory of Strengths (VIA-IS) and Their Relationships to Life Satisfaction

Psychological Reports ◽

10.1177/0033294118779198 ◽

2018 ◽

Vol 122 (3) ◽

pp. 1167-1188 ◽

Cited By ~ 5

Author(s):

Mercedes Ovejero Bruna ◽

Andreea C. Brabete ◽

Jesús M. Alvarado Izquierdo

Keyword(s):

Life Satisfaction ◽

Test Scores ◽

Character Strengths ◽

Meta Analysis ◽

Self Regulation ◽

Reliability Estimation ◽

Assessment Instrument ◽

Sample Type ◽

Reliability Generalization ◽

The Relationship

Reliable test scores are essential to interpret the results obtained in statistical analyses correctly. In this study, we used the Values in Action Inventory of Strengths (VIA-IS) as an example of a widely applied assessment instrument to analyze its metric quality in what is known as reliability generalization (RG). In addition, we conducted a meta-analysis of the correlations between character strengths and life satisfaction to examine the potential relationship between the reliability of test scores and the intensity of these correlations. The overall variability of alpha coefficients supports the argument that reliability is sample dependent. Indeed, there were statistically significant mean reliability differences for scores across the 24 scales, with the highest level of reliability observed for Creativity and the lowest for scores on Self-regulation. Significant moderators such as the standard deviation of the scores and the sample type contribute to understand the high variability observed in the reliability estimation. The second meta-analysis showed that Zest, Hope, Gratitude, Curiosity, and Love were the character strengths that were highly related to life satisfaction, while Modesty and Prudence were less related to life satisfaction. Furthermore, the high heterogeneity between samples might be an indicator of the relationship between the variability of reliability of character strengths' scores and the intensity of their correlations with life satisfaction. Those character strengths with high-potential RG are related or unrelated to life satisfaction, whereas character strengths with less-potential RG showed unstable correlation patterns. The results of both studies point out the role of the relationship between the reliability of test scores and substantive studies, such as Pearson's correlations meta-analysis.

Download Full-text