Psychometric Evaluation of the Cybersecurity Concept Inventory

We present a psychometric evaluation of a revised version of the Cybersecurity Concept Inventory (CCI) , completed by 354 students from 29 colleges and universities. The CCI is a conceptual test of understanding created to enable research on instruction quality in cybersecurity education. This work extends previous expert review and small-scale pilot testing of the CCI. Results show that the CCI aligns with a curriculum many instructors expect from an introductory cybersecurity course, and that it is a valid and reliable tool for assessing what conceptual cybersecurity knowledge students learned.

Psychometrics in Behavioral Software Engineering: A Methodological Introduction with Guidelines

A meaningful and deep understanding of the human aspects of software engineering (SE) requires psychological constructs to be considered. Psychology theory can facilitate the systematic and sound development as well as the adoption of instruments (e.g., psychological tests, questionnaires) to assess these constructs. In particular, to ensure high quality, the psychometric properties of instruments need evaluation. In this article, we provide an introduction to psychometric theory for the evaluation of measurement instruments for SE researchers. We present guidelines that enable using existing instruments and developing new ones adequately. We conducted a comprehensive review of the psychology literature framed by the Standards for Educational and Psychological Testing. We detail activities used when operationalizing new psychological constructs, such as item pooling, item review, pilot testing, item analysis, factor analysis, statistical property of items, reliability, validity, and fairness in testing and test bias. We provide an openly available example of a psychometric evaluation based on our guideline. We hope to encourage a culture change in SE research towards the adoption of established methods from psychology. To improve the quality of behavioral research in SE, studies focusing on introducing, validating, and then using psychometric instruments need to be more common.

Family Sense of Coherence Scale: A Confirmatory Factor Analysis in a Portuguese Sample

Background: Family sense of coherence (FSOC) can be defined as the cognitive map of a family that enables the family to deal with stress during their lifetime. FSOC is the degree to which a family perceives family life as comprehensible, manageable, and meaningful. To the best of our knowledge, few studies have used this scale, and very few have evaluated FSOC Scale psychometric properties.Objective: This study aimed to evaluate the psychometric properties of the original FSOC Scale in a sample of Portuguese caregivers of children aged between 10 and 15 years.Methods: A total of 329 caregivers completed a sociodemographic questionnaire and the FSOC Scale. Analyses were performed to evaluate the factor structure of the FSOC Scale with 26 items as well as composite reliability, internal consistency, convergent-related validity, and discriminant-related validity of the scale scores.Results: The findings supported a three-factor solution for a 13-item version that maintains the original FSOC Scale structure. The three FSOC dimensions presented a good fit to the data. Cronbach’s alpha, composite reliability, and convergent-related validity were considered very good for the FSOC Scale (α = 0.956; CR = 0.974; AVE = 0.689). No evidence of discriminant-related validity was found for the dimensions of FSOC.Conclusion: The findings support the use of the Portuguese FSOC Scale for research and clinical purposes with Portuguese caregivers. Future research is necessary to further develop a European Portuguese version of the FSOC Scale.Implications: This study provides a psychometric evaluation of FSOC Scale characteristics in a Portuguese sample. The results are helpful for clinicians and family therapists who work with families since it could help them to assess the resources of families and their ability to cope with adversity and enhance their strengths.

Development and Psychometric Evaluation of the Gender Identity Scale for Transgender Women in China

Transgender women are an important subgroup of the transgender umbrella and have their own unique gender identity. This article aimed to understand and measure the latent concept of gender identity among Chinese transgender women from a multi-dimensional perspective. Through a two-phase, iterative scale development process, we developed the Gender Identity Scale for Transgender Women (GIS-TW) in Chinese. Literature reviews, expert consultations, and focus groups constitute phrase 1 of the study, which resulted in the first version of GIS-TW with 30 items. In phrase 2, exploratory factor analysis on a sample of 244 Chinese transgender women revealed a six-factor solution across the 22 items. The Bem Sex Role Inventory was included to test for convergent validity, and the Rosenberg Self-Esteem Scale was used to test discriminant validity. Then we conducted the confirmatory factor analysis with an independent sample of 420 Chinese transgender women, which produced the final version of GIS-TW with 21 items. The internal consistency (Cronbach’s alpha = 0.71–0.87) and test-retest stability (r = 0.73–0.87) of each factor was good. In conclusion, the GIS-TW is a reliable and valid psychometric tool for the assessment of Chinese transgender women’s gender identity. Future application of the scale will help transgender women obtain better gender confirmative interventions.