Development and Testing of an Instrument for Summative Assessment of Practical Skill Performance: A Generalizability Theory Approach

Background and PurposeMany newly graduated nurses lack proficiency in practical skill performance. Presently, nursing students’ practical skill is assessed by summative instruments with overarching items. The purpose of this study was to develop a more detailed instrument to use in summative assessment of nursing students’ practical skill performance and to assess its psychometric properties.MethodsA 50-item instrument was developed. Video-recorded performances were rated by experienced clinical supervisors. A multifacet measurement design was developed. Relevant parameters were estimated by generalizability analysis.ResultsFindings indicated that error of measurement were mainly caused by raters far more than by items. Conclusions: The present study suggested that summative assessment in realistic settings may not apply one rater only. Two to three/four raters appear necessary to dependably measure most skills.

Download Full-text

Improving Clinical Nurses’ Development of Supervision Skills through an Action Learning Approach

Nursing Research and Practice ◽

10.1155/2020/9483549 ◽

2020 ◽

Vol 2020 ◽

pp. 1-10

Author(s):

Irene Sommer ◽

Karin Larsen ◽

Carsten M. Nielsen ◽

Britta V. Stenholt ◽

Ida Torunn Bjørk

Keyword(s):

Nursing Students ◽

Clinical Education ◽

Large Scale ◽

Action Learning ◽

Implementation Process ◽

Practical Skill ◽

Clinical Supervisors ◽

Learning Group ◽

Skill Performance ◽

The Right

The aim of this study was to investigate action learning as an implementation method in a large-scale project with many participants in several autonomous and geographically spread groups. The focus of the implementation was the Model of Practical Skill Performance as a learning and supervision tool in the clinical education of nursing students. Nineteen action learning groups were established, and a total of 129 clinical supervisors and 13 facilitators were involved. To evaluate the implementation process, qualitative data were generated through three focus group interviews, questionnaires, and notes. Data illuminate clinical supervisors’ perceptions of value, impact, and sustainability when they participate in an action learning group to become familiar with the Model of Practical Skill Performance. The deductive data analysis was guided by central concepts from action learning. Action learning proved to be an engaging and effective tool in the implementation where the main strength seemed to be the autonomous local group supporting collective reflections on actions. Clinical supervisors had the right competences to adopt a reflective process-oriented approach, which is the hallmark of action learning. This study shows the necessity of collaboration between stakeholders in practice, education, and management to implement large-scale projects in clinical practice. The findings imply that managers should choose participants on the basis of their motivation and their voluntary wish to participate and that nurses’ immersion in the project over time aids implementation.

Download Full-text

A generalizability theory study of optimal measurement design for a summative assessment of English/Chinese consecutive interpreting

Language Testing ◽

10.1177/0265532218809396 ◽

2018 ◽

Vol 36 (3) ◽

pp. 419-438 ◽

Cited By ~ 2

Author(s):

Chao Han

Keyword(s):

Generalizability Theory ◽

Summative Assessment ◽

The Other ◽

Target Language ◽

Complex Nature ◽

High Stakes ◽

Optimal Measurement ◽

Language Quality ◽

Measurement Design ◽

Consecutive Interpreting

Summative assessment of interpretation is widely conducted in interpreting courses/programs to inform high-stakes decision making, such as the selection, certification, and conferral of academic degrees. Yet there has been very limited empirical research to investigate the score dependability of summative interpretation assessment. The present study therefore sets out to explore the optimal measurement design(s) for a locally created summative assessment of English/Chinese consecutive interpretation, based on multiple fully crossed generalizability studies. Major findings include the following: (a) overall the raters behaved more consistently by using the information completeness (InfoCom) scale rather than the fluency of delivery (FluDel) or target language quality (TLQual) scales; (b) the raters displayed greater variability in evaluating the Chinese-to-English interpretation rather than the English-to-Chinese interpretation; (c) although adding tasks worked more effectively in raising score dependability than using additional raters for the InfoCom ratings in the English-to-Chinese interpretation, the pattern was reversed for the other observations; and (d) two potentially optimal designs were identified for the English-to-Chinese direction, and one design for the other direction. These results are discussed, highlighting the complex nature of relationships among the assessment criterion, the interpreting directionality, the raters’ dominant language and score dependability, together with the need to ensure score dependability for summative interpretation assessment.

Download Full-text

A Generalizability Theory Approach to If...Then..., Situation-Behavior Relations

PsycEXTRA Dataset ◽

10.1037/e566962012-512 ◽

2009 ◽

Author(s):

James D. Kloet ◽

Brian Lakey

Keyword(s):

Generalizability Theory ◽

Theory Approach

Download Full-text

Applying generalizability theory to examine assessments of subjective cognitive complaints: whose reports should we rely on – participant versus informant?

International Psychogeriatrics ◽

10.1017/s1041610221000363 ◽

2021 ◽

pp. 1-11

Author(s):

Q. C. Truong ◽

C. Choo ◽

K. Numbers ◽

A. G. Merkin ◽

H. Brodaty ◽

...

Keyword(s):

Predominantly White ◽

Generalizability Theory ◽

The Elderly ◽

Community Dwelling ◽

Assessment Tools ◽

Sample Population ◽

Cognitive Complaints ◽

Longitudinal Measurement ◽

Measurement Design ◽

Subjective Cognitive Complaints

ABSTRACT Objectives: This study aimed to apply the generalizability theory (G-theory) to investigate dynamic and enduring patterns of subjective cognitive complaints (SCC), and reliability of two widely used SCC assessment tools. Design: G-theory was applied to assessment scales using longitudinal measurement design with five assessments spanning 10 years of follow-up. Setting: Community-dwelling older adults aged 70–90 years and their informants, living in Sydney, Australia, participated in the longitudinal Sydney Memory and Ageing Study. Participants: The sample included 232 participants aged 70 years and older, and 232 associated informants. Participants were predominantly White Europeans (97.8%). The sample of informants included 76 males (32.8%), 153 females (65.9%), and their age ranged from 27 to 86 years, with a mean age of 61.3 years (SD = 14.38). Measurements: The Memory Complaint Questionnaire (MAC-Q) and the Informant Questionnaire on Cognitive Decline in the Elderly (IQCODE). Results: The IQCODE demonstrated strong reliability in measuring enduring patterns of SCC with G = 0.86. Marginally acceptable reliability of the 6-item MAC-Q (G = 0.77–0.80) was optimized by removing one item resulting in G = 0.80–0.81. Most items of both assessments were measuring enduring SCC with exception of one dynamic MAC-Q item. The IQCODE significantly predicted global cognition scores and risk of dementia incident across all occasions, while MAC-Q scores were only significant predictors on some occasions. Conclusions: While both informants’ (IQCODE) and self-reported (MAC-Q) SCC scores were generalizable across sample population and occasions, self-reported (MAC-Q) scores may be less accurate in predicting cognitive ability and diagnosis of each individual.

Download Full-text

Supplemental Material for Interrater Reliability for Multilevel Data: A Generalizability Theory Approach

Psychological Methods ◽

10.1037/met0000391.supp ◽

2021 ◽

Keyword(s):

Interrater Reliability ◽

Generalizability Theory ◽

Theory Approach ◽

Multilevel Data

Download Full-text

Socializing for Authentic Caring Engagement in Nursing Practice: Nursing Student Moral Development in Preceptorship

Canadian Journal of Nursing Research ◽

10.1177/0844562118809258 ◽

2018 ◽

Vol 51 (2) ◽

pp. 63-71 ◽

Cited By ~ 3

Author(s):

Jill Vihos ◽

Florence Myrick ◽

Olive Yonge

Keyword(s):

Nursing Students ◽

Moral Development ◽

Nursing Student ◽

Nursing Practice ◽

Moral Identity ◽

Theory Approach ◽

Moral Issues ◽

Undergraduate Nursing ◽

Patient’S Experience ◽

Authentic Caring

Background The purpose of this study was to explore the basic psychosocial process of undergraduate nursing student moral development in clinical preceptorship. Method A grounded theory approach was used to explore the process within the context of clinical practice and the student–preceptor–faculty member relationship. Results Socializing for authentic caring engagement in nursing practice emerged from the data as the basic psychosocial process of nursing student moral development in preceptorship. This process included four key categories: (a) distinguishing nursing and moral identity in practice, (b) learning to recognize the patient’s experience, (c) identifying moral issues in practice and creating meaning of practice encounters, and (d) becoming an advocate and reconciling moral issues in practice. Conclusion Findings emerging from this study illustrate the processes of how nursing students work through moral issues and the role of faculty and preceptors in engaging students with moral encounters in the context of preceptorship.

Download Full-text

Nursing Graduates' Perceptions of Their Undergraduate Clinical Placement

International Journal of Nursing Education Scholarship ◽

10.2202/1548-923x.1276 ◽

2007 ◽

Vol 4 (1) ◽

Cited By ~ 59

Author(s):

Jackie A Hartigan-Rogers ◽

Shelley L Cobbett ◽

Mardi A Amirault ◽

Martha E Muise-Davis

Keyword(s):

Nursing Education ◽

Nursing Students ◽

Semantic Analysis ◽

Nursing Graduates ◽

Clinical Experiences ◽

Graduate Nurses ◽

Clinical Placements ◽

Newly Graduated Nurses ◽

Nurses Perceptions ◽

Supportive Learning Environment

Student clinical practice is a significant and essential part of nursing education. Accordingly, clinical placements need to be positive and valuable. The purpose of this study was to describe newly-graduated nurses' perceptions of their student clinical intersession placements and how these placements impacted their functioning as graduate nurses. Inductive semantic analysis of the data revealed four themes: developing nursing skills and knowledge; preparing for future employment; experiencing supportive relationships; and experiencing realities of work-life. Educators must ensure that clinical placements are in an environment that is conducive to learning and promotes the personal and professional development of nursing students. Positive clinical experiences are more likely related to how valued and supported students feel than the physical aspects of a placement; thus, it is imperative educators assess and focus on providing placements that offer a supportive learning environment.

Download Full-text