Managing Heterogeneity of Variance in Studies of Reliability Generalization With Alpha Coefficients

Methodology ◽  
2012 ◽  
Vol 8 (2) ◽  
pp. 71-80 ◽  
Author(s):  
Juan Botella ◽  
Manuel Suero
Keyword(s):  
Reliability Estimation ◽  
Biased Sampling ◽  
Reliability Generalization ◽  
Alpha Coefficient ◽  
Range Restriction ◽  
Sampling Schemes ◽  
Cronbach's Alpha ◽  
Heterogeneous Variances ◽  
Heterogeneity Of Variance ◽  
Meta Analyses

In Reliability Generalization (RG) meta-analyses, the importance of bearing in mind the problems of range restriction or biased sampling and their influence on reliability estimation has often been highlighted. Nevertheless, the presence of heterogeneous variances in the included studies has been diagnosed in a subjective way and has not been taken into account in later analyses. Procedures to detect the presence of a variety of sampling schemes and to manage them in the analyses are proposed. The procedures are further explained with an example, by applying them to 25 estimates of Cronbach’s alpha coefficient in the Hamilton Scale for Depression.

scholarly journals A Reliability Generalization Study for a Multidimensional Loneliness Scale

2015 ◽  
Vol 31 (4) ◽  
pp. 294-301 ◽  
Author(s):  
Marlies Maes ◽  
Wim Van den Noortgate ◽  
Luc Goossens
Keyword(s):  
Children And Adolescents ◽  
Good Option ◽  
Future Research ◽  
Reliability Generalization ◽  
Cronbach’S Alpha ◽  
Cronbach's Alpha ◽  
Loneliness Scale ◽  
Meta Analyses

Abstract. Research on the average reliability and factors that affect the reliability of loneliness scales has been restricted to unidimensional measures. A reliability generalization (RG) study was conducted for a multidimensional loneness measure, that is, the Loneliness and Aloneness Scale for Children and Adolescents (LACA). Multilevel meta-analyses were performed on 79 studies that comprised 92 samples (for a total of 41,076 participants). Average reliability (Cronbach’s alpha) across samples was good (i.e., .80 or above) for all four subscales. Studies with higher sampling quality yielded slightly higher alphas for one of the subscales (i.e., Parent-related loneliness). For adolescents, as compared to children, alphas were somewhat lower for three of the four subscales and higher for the Affinity for aloneness subscale. Suggestions for future research are outlined. From a reliability perspective, the LACA is a good option for researchers who want to use a multidimensional loneliness measure with children and adolescents.

scholarly journals Polish Adaptation of the Italian Spine Youth Quality of Life Questionnaire

2021 ◽  
Vol 10 (10) ◽  
pp. 2081
Author(s):  
Edyta Kinel ◽  
Krzysztof Korbel ◽  
Piotr Janusz ◽  
Mateusz Kozinoga ◽  
Dariusz Czaprowski ◽  
...  
Keyword(s):  
Quality Of Life ◽  
Internal Consistency ◽  
Spinal Deformity ◽  
Cultural Adaptation ◽  
Life Questionnaire ◽  
Alpha Coefficient ◽  
Cronbach’S Alpha ◽  
Quality Of Life Questionnaire ◽  
Cronbach's Alpha

The study aimed to carry on the process of the cultural adaptation of the Italian Spine Youth Quality of Life Questionnaire (ISYQOL) into Polish (ISYQOL-PL). The a priori hypothesis was: the ISYQOL-PL questionnaire is reliable and appropriate for adolescents with a spinal deformity. Fifty-six adolescents (mean age 13.8 ± 1.9) with idiopathic scoliosis (AIS) with a mean Cobb angle 29.1 (±9.7) and two with Scheuermann juvenile kyphosis (SJK) with a kyphosis angle 67.5 (±17.7) degrees were enrolled. All patients had been wearing a corrective TLSO brace for an average duration of 2.3 (±1.8) years. The Institutional Review Board approved the study. The cross-cultural adaptation of the ISYQOL-PL was performed following the guidelines set up by the International Quality of Life Assessment Project. The reliability was assessed using internal consistency (the Cronbach’s alpha coefficient) and test–retest reliability (intraclass correlation coefficient ICC2.1, CI = 95%); moreover, floor and ceiling effects were calculated. The internal consistency was satisfactory (Cronbach’s alpha coefficient 0.8). The test–retest revealed high reliability with the value of ICC2.1 for the entire group 0.90, CI (0.84 to 0.94). There was neither floor nor ceiling effect for the ISYQOL-PL overall score. The ISYQOL-PL is reliable and can be used in adolescents with spinal deformity.

scholarly journals Adaptação transcultural do questionário EFFECT para português brasileiro

2021 ◽  
Vol 45 (3) ◽  
Author(s):  
Lourrany Borges Costa ◽  
Shamyr Sulyvan de Castro ◽  
Diovana Ximenes Cavalcante Dourado ◽  
Bruna Soares Praxedes ◽  
Thayná Custódio Mota ◽  
...  
Keyword(s):  
Teacher Evaluation ◽  
Internal Consistency ◽  
Clinical Teaching ◽  
Clinical Medicine ◽  
Brazilian Portuguese ◽  
Expert Committee ◽  
Alpha Coefficient ◽  
Cronbach’S Alpha ◽  
Cronbach's Alpha ◽  
Back Translation

Abstract: Introduction: Clinical teaching is based on a real work environment, in professional practice settings, such as health services and units, under the supervision of the preceptor. Providing medical teachers with an assessment of their teaching skills is a powerful tool for improving clinical learning for students in training. In this context, the EFFECT (Evaluation and Feedback for Effective Clinical Teaching) questionnaire was developed by Dutch researchers in 2012 for teacher evaluation, being validated based on the literature about medical teaching in the workplace and incorporates the skills of the Canadian competency-based medical curriculum. Objective: To translate and cross-culturally adapt into Brazilian Portuguese and to validate the EFFECT questionnaire for teacher evaluation by Medical students. Method: Cross-cultural adaptation with the following steps: initial translation of the English version, synthesis of translated versions, back-translation, creation of a consensual version in Brazilian Portuguese, with adaptation, review, and analysis of content validity by an expert committee, pre-test with retrospective clarification interview, and reliability analysis by factorial analysis and internal consistency test (Cronbach’s alpha coefficient). Result: In the translation and back-translation stages, the disagreements were related to the use of synonyms and none of the items were modified in terms of their understanding, but in terms of adaptation into the Brazilian context. The evaluation of the expert committee showed the versions maintained the semantic and idiomatic equivalences of the content. Eighty-nine students participated in the pre-test. The internal consistency of the EFFECT questionnaire in Brazilian Portuguese was excellent for all domains, with Cronbach’s alpha coefficient ranging from 0.82 to 0.94. Conclusion: The translated and adapted version of the EFFECT questionnaire into Brazilian Portuguese is equivalent to the original instrument and has evidence of high validity and reliability, being able to constitute a national tool to evaluate the efficiency of clinical medicine teaching.

scholarly journals THE HOMOGENEITY OF SCALES OF INTELLECTUAL ABILITIES: PSYCHOMETRIC ANALYSIS

2021 ◽  
pp. 144-169
Author(s):  
Anatoly N. Krichevets ◽  
Alexey A. Korneev ◽  
K.V. Sugonyaev
Keyword(s):  
Factor Analysis ◽  
Confirmatory Factor Analysis ◽  
Detailed Analysis ◽  
Alpha Coefficient ◽  
Cronbach’S Alpha ◽  
Testing Procedures ◽  
Cronbach's Alpha ◽  
Psychometric Instruments ◽  
Confirmatory Factor ◽  
Item Level

Relevance. Nowadays the researchers commonly use a limited set of standard procedures and statistical coefficients when develop psychometric instruments and investigate their structure. The routine using of such procedures without taking into account the specific features of the psychometric scales can lead to incomplete or even inadequate results. In this context detailed consideration of the structure of psychometric instruments seems to be important and it may demand various non-standard ways of statistical analysis. Objectives. To conduct detailed analysis of the results of two intelligent subtests at the item level and to assess the sufficiency and adequacy of using standard methods for estimation of reliability and structural validity for these subtests. Methods. We analyze the data collected in intelligence testing of a large sample of respondents (11335 young adults). The respondents passed the KR-3 battery. In this study we examine in detail the structure of the subtests “Syllogisms” and “Analogies”. Specifically, we estimated the reliability of the scales by the Cronbach’s alpha coefficient, and the structure at the item level using the confirmatory factor analysis. Results and conclusions. Estimation of the reliability of the scales by Cronbach’s alpha coefficient showed importance of taking into account the time limitation, which is commonly used in intelligence tests. On the other hand, a detailed analysis of each subtest items made it possible to find out an additional factor which was not originally proposed in the factor structure. This is factor of higher-order abilities of abstract analysis, whilst the subtest originally aimed at estimation of the special abilities. Confirmatory factor analysis showed improvement of fit when this factor was added. The results allow to conclude that the researcher may miss the important properties of scales if not making a detailed analysis of testing procedures and the structure of subtest at the item level, and so may draw incomplete or inadequate conclusions about their psychometric properties

scholarly journals Polish adaptation of the Iowa Satisfaction with Anaesthesia Scale

2016 ◽  
Vol 15 (4) ◽  
pp. 28-34 ◽  
Author(s):  
Danuta Dyk ◽  
Dorota Ozga ◽  
Aleksandra Gutysz-Wojnicka ◽  
Bogumił Lewandowski ◽  
Marek Wojtaszek
Keyword(s):  
Maxillofacial Surgery ◽  
Multiple Choice Questions ◽  
Alpha Coefficient ◽  
Cronbach’S Alpha ◽  
Cronbach's Alpha ◽  
Clinical Ward ◽  
Polish Version ◽  
Continue Research ◽  
Three Stages ◽  
Polish Adaptation

Abstract Introduction. The paper presents the adaptation of the Iowa Satisfaction with Anesthesia Scale to the conditions in Polish hospitals.Methods. The process of cultural adaptation of the Iowa Satisfaction with Anesthesia Scale was divided into three stages. The first stage was to formulate the Polish version of the ISAS. Secondly, the scale was used in the group of patients who were diagnosed with a maxillofacial injury and underwent surgical procedure. The final stage evaluated the psychometric features of the Polish version of the ISAS.Results. 195 questionnaires were completed correctly and they were subject to further analysis. The questionnaires were collected from patients residing at the Maxillofacial Surgery Clinical Ward who were operated under a general anaesthesia due to a maxillofacial trauma. The average result of the satisfaction with anaesthesia in a studied group was 0.77 (scale from -3 to +3) M=2, SD=2.41. The Cronbach’s alpha coefficient calculated for 11 multiple choice questions was 0.598. It would be possible to increase the Cronbach’s alpha coefficient if question no. 1 (I vomited or felt nauseous) was eliminated from the test (maximum 0.601). Every item on the scale had significant influence on the general result (p=0.0001).Conclusions. Polish version of the Iowa Satisfaction with Anesthesia Scale meets the criteria of the psychometric equivalence with the original version. It is necessary to continue research using Polish version of the ISAS to provide a constant verification of reliability and accuracy of the research tool.

Reliability Generalization as a Seal of Quality of Substantive Meta-Analyses: The Case of the VIA Inventory of Strengths (VIA-IS) and Their Relationships to Life Satisfaction

2018 ◽  
Vol 122 (3) ◽  
pp. 1167-1188 ◽  
Author(s):  
Mercedes Ovejero Bruna ◽  
Andreea C. Brabete ◽  
Jesús M. Alvarado Izquierdo
Keyword(s):  
Life Satisfaction ◽  
Test Scores ◽  
Character Strengths ◽  
Meta Analysis ◽  
Self Regulation ◽  
Reliability Estimation ◽  
Assessment Instrument ◽  
Sample Type ◽  
Reliability Generalization ◽  
The Relationship

Reliable test scores are essential to interpret the results obtained in statistical analyses correctly. In this study, we used the Values in Action Inventory of Strengths (VIA-IS) as an example of a widely applied assessment instrument to analyze its metric quality in what is known as reliability generalization (RG). In addition, we conducted a meta-analysis of the correlations between character strengths and life satisfaction to examine the potential relationship between the reliability of test scores and the intensity of these correlations. The overall variability of alpha coefficients supports the argument that reliability is sample dependent. Indeed, there were statistically significant mean reliability differences for scores across the 24 scales, with the highest level of reliability observed for Creativity and the lowest for scores on Self-regulation. Significant moderators such as the standard deviation of the scores and the sample type contribute to understand the high variability observed in the reliability estimation. The second meta-analysis showed that Zest, Hope, Gratitude, Curiosity, and Love were the character strengths that were highly related to life satisfaction, while Modesty and Prudence were less related to life satisfaction. Furthermore, the high heterogeneity between samples might be an indicator of the relationship between the variability of reliability of character strengths' scores and the intensity of their correlations with life satisfaction. Those character strengths with high-potential RG are related or unrelated to life satisfaction, whereas character strengths with less-potential RG showed unstable correlation patterns. The results of both studies point out the role of the relationship between the reliability of test scores and substantive studies, such as Pearson's correlations meta-analysis.

scholarly journals Evaluating Reliability of Theory of Planned Behaviour Questionnaire for Withdrawal of Divorce Petition

2018 ◽  
Vol 6 (8) ◽  
pp. 1512-1516 ◽  
Author(s):  
Nahid Ardian ◽  
Seyed Alireza Afshani ◽  
Mohammad Ali Morowatisharifabad ◽  
Seyed Saeid Mazloomi Mahmoodabad ◽  
Ali Akbar Vaezi ◽  
...  
Keyword(s):  
Qualitative Study ◽  
Content Validity ◽  
Theory Of Planned Behaviour ◽  
Validity And Reliability ◽  
Alpha Coefficient ◽  
Cronbach’S Alpha ◽  
Planned Behaviour ◽  
Cronbach's Alpha ◽  
The Face ◽  
Suitable Framework

BACKGROUND: Given the increased rate of divorce, it is important to analyse the characteristics of divorce applicants. The theory of planned behaviour (TPB) can provide a suitable framework to predict, explain, and/or change the behaviours. In Iran, no instrument can be found, based on health education models, to investigate divorce petition filing as a behaviour.AIM: This study was conducted to design a questionnaire on withdrawal of divorce petition based on the TPB and estimate its validity and reliability.MATERIAL AND METHODS: A qualitative study was conducted in 27 participants involved in the divorce process using directed content analysis. The face and content validity of 58 items, drawn from the qualitative study, were evaluated by 10 experts. The reliability was estimated using Cronbach’s alpha coefficient. The SPSS version 16 was used to analyse data.RESULTS: Estimates of the face and content validity (quantitative and qualitative), revealed that of the 58 items, 48 were valid based on four of the constructs of the TPB. Cronbach’s alpha coefficient was also derived greater than 0.6.CONCLUSION: The designed questionnaire, whose validity and reliability was confirmed in this study, can be used in similar studies. However, the social and cultural differences and their related effects should be considered.

scholarly journals Validation of the Spanish Version of Newcastle Stroke-Specific Quality of Life Measure (NEWSQOL)

2020 ◽  
Vol 17 (12) ◽  
pp. 4237
Author(s):  
Concepción Soto-Vidal ◽  
Soraya Pacheco-da-Costa ◽  
Victoria Calvo-Fuente ◽  
Sara Fernández-Guinea ◽  
Carlos González-Alted ◽  
...  
Keyword(s):  
Quality Of Life ◽  
Construct Validity ◽  
Response Rate ◽  
Spanish Version ◽  
Psychometric Validation ◽  
Alpha Coefficient ◽  
Cronbach’S Alpha ◽  
Life Measure ◽  
Cronbach's Alpha

Background: Stroke causes a wide variety of clinical manifestations that may have a negative impact on quality of life. Therefore, it is very important to use specific instruments for measuring quality of life in individuals who suffered a stroke. The aim of this study was to develop a psychometrically validated Spanish version of the Newcastle stroke-specific quality of life measure (NEWSQOL). Methods: A psychometric validation of the Spanish version of the NEWSQOL questionnaire was carried out in 159 patients. The reliability (intraclass correlation coefficient and Cronbach’s alpha coefficient), validity (factorial analysis and Spearman’s coefficient), feasibility (response rate), and the ceiling and floor effects were calculated. Results: Internal consistency showed that Cronbach’s alpha coefficient was 0.93. The test–retest reliability was high or excellent for all domains (range 0.71–0.97 p < 0.001). The response rate of the questionnaire was 100% and the average administration time was 20.5 (±7.2) min. No ceiling effect was detected and two domains (pain and vision) may have a significant potential for floor effect. Construct validity showed that all the variables are important enough to keep them all in the questionnaire. Concerning convergent construct validity, a high correlation was found with the Nottingham Health Profile, the Barthel Index, and the Modified Rankin Scale. Conclusion: The Spanish version of the NEWSQOL questionnaire is reliable, valid, and feasible to evaluate quality of life in the Spanish population.

Response Rates for Surveys Completed With Paper-and-Pencil and Computers: Using Meta-Analysis to Assess Equivalence

2018 ◽  
Vol 37 (5) ◽  
pp. 649-668 ◽  
Author(s):  
Arne Weigold ◽  
Ingrid K. Weigold ◽  
Sara N. Natera
Keyword(s):  
Meta Analysis ◽  
Response Rates ◽  
Self Report ◽  
Independent Effect ◽  
Equivalence Testing ◽  
Testing Procedures ◽  
Heterogeneity Of Variance ◽  
Large Heterogeneity ◽  
Paper And Pencil ◽  
Meta Analyses

The increasing number of self-report surveys being collected using computers has led to a body of literature examining the response rates for computerized surveys compared with the more traditional paper-and-pencil method. However, results from individual studies have been inconsistent, and the meta-analyses available on this topic have included studies from a restricted range of years and did not use proper statistical procedures for examining comparability. Consequently, we conducted a meta-analysis with 96 independent effect sizes spanning over two decades of studies; we also assessed potential moderators. Comparability was determined using confidence interval equivalence testing procedures. The meta-analysis indicated nonequivalence, with those in the paper-and-pencil condition being almost twice as likely to return surveys as those in the computer condition. There was large heterogeneity of variance, and 11 of the 18 potential moderators were significant. Two meta-regressions yielded only two significant unique moderators: population and type of measure. Results highlighted issues within the response rate literature that can be addressed in future studies, as well as provided an example of using equivalence testing in meta-analyses.

scholarly journals Empathy in Undergraduate Medical Students of Bangladesh: Psychometric Analysis and Differences by Gender, Academic Year, and Specialty Preferences

2014 ◽  
Vol 2014 ◽  
pp. 1-7 ◽  
Author(s):  
Asma Mostafa ◽  
Rozina Hoque ◽  
Mohammad Mostafa ◽  
Md. Mashud Rana ◽  
Faisal Mostafa
Keyword(s):  
Medical Students ◽  
Clinical Training ◽  
Medical Curriculum ◽  
Principal Component Factor Analysis ◽  
Validity And Reliability ◽  
Alpha Coefficient ◽  
Undergraduate Medical Students ◽  
Cronbach’S Alpha ◽  
Medical College ◽  
Cronbach's Alpha

Empathy is considered to be associated with better patient compliance, satisfaction, and clinical outcomes. The aim of the study is to measure and examine empathy among a sample of undergraduate medical students of Bangladesh. It was a cross-sectional study and all the medical students of first through fifth year enrolled at Chattagram Maa-O-Shishu Hospital Medical College during the study period of 2014 were surveyed. Participants anonymously completed the Jefferson Scale of Empathy Medical Student version translated into Bengali language, a valid and reliable 20-item self-administered questionnaire. Principal component factor analysis with varimax rotation and Cronbach’s alpha coefficient were calculated to check validity and reliability of the scale. ANOVA was used to examine the differences in empathy between gender, academic years, and specialty preferences. The mean empathy score was 110.41 ± 13.59. Cronbach’s alpha coefficient was 0.88. There were significant associations between gender and empathy scores. The level of empathy in medical students gradually increases after clinical training in medical college. A nonsignificant difference was found between empathy scores and specialty preferences. It is suggested that the medical curriculum in Bangladesh should include more extensive program to promote empathy and other humanistic values among the medical students.

Sign in / Sign up

Export Citation Format

Share Document