Measuring hygiene competence: the picture-based situational judgement test HygiKo

Background With the onset of the COVID-19 pandemic at the beginning of 2020, the crucial role of hygiene in healthcare settings has once again become very clear. For diagnostic and for didactic purposes, standardized and reliable tests suitable to assess the competencies involved in “working hygienically” are required. However, existing tests usually use self-report questionnaires, which are suboptimal for this purpose. In the present study, we introduce the newly developed, competence-oriented HygiKo test instrument focusing health-care professionals’ hygiene competence and report empirical evidence regarding its psychometric properties. Methods HygiKo is a Situational Judgement Test (SJT) to assess hygiene competence. The HygiKo-test consists of twenty pictures (items), each item presents only one unambiguous hygiene lapse. For each item, test respondents are asked (1) whether they recognize a problem in the picture with respect to hygiene guidelines and, (2) if yes, to describe the problem in a short verbal response. Our sample comprised n = 149 health care professionals (79.1 % female; age: M = 26.7 years, SD = 7.3 years) working as clinicians or nurses. The written responses were rated by two independent raters with high agreement (α > 0.80), indicating high reliability of the measurement. We used Item Response Theory (IRT) for further data analysis. Results We report IRT analyses that show that the HygiKo-test is suitable to assess hygiene competence and that it allows to distinguish between persons demonstrating different levels of ability for seventeen of the twenty items), especially for the range of low to medium person abilities. Hence, the HygiKo-SJT is suitable to get a reliable and competence-oriented measure for hygiene-competence. Conclusions In its present form, the HygiKo-test can be used to assess the hygiene competence of medical students, medical doctors, nurses and trainee nurses in cross-sectional measurements. In order to broaden the difficulty spectrum of the current test, additional test items with higher difficulty should be developed. The Situational Judgement Test designed to assess hygiene competence can be helpful in testing and teaching the ability of working hygienically. Further research for validity is needed.

Indirect skill assessment using AI technology

In the new context of transitional, mobilised and globalised labour markets, an urgent need has emerged for meaningful assessment tools, methods and techniques to measure and recognize the workers’ skills. This paper aims to present a novel approach in skill assessment developed under NADINE H2020 project, the indirect skill assessment, as an alternative or supplement to the traditional selection methods. AI technology is utilised to facilitate the indirect skill assessment via two agnostic content serious games (Tetris and 2048), based on the evaluation of an individual’s performance in playing a game outside a situational judgement test (SJT) context. Novel datasets have been developed, comprised of game sessions and the corresponding skill assessments of the players through validated psychometric questionnaires, which were the basis for the algorithm training that would provide the estimation of a player’s skillset. The trained neural models for both games proved to have strong skill assessment capabilities, indicating that there is indeed a correlation between a person’s action sequence and his/her different skills.

191 Performance at Medical School and Foundation Programme Selection Can Predict Success in The MRCS Examination

Background Identifying factors that predict success in the Membership of the Royal College of Surgeons (MRCS) examination can aid trainees’ exam preparation and help deaneries identify candidates likely to require additional support. We assessed whether performance at Medical School and UK Foundation Programme selection (Educational Performance Measurement (EPM) and Situational Judgement Test (SJT)) predicted MRCS success. Method We analysed data from the UKMED Database for UK graduates who had attempted MRCS Part A (n = 1,975) and Part B (n = 630) between 2013-2017. Univariate analysis examined the relationship between performance and the likelihood of passing MRCS at first attempt. Logistic regression identified independent predictors of MRCS success. Results Each additional EPM decile increased the chances of passing MRCS by 52% for Part A (odds ratio 1.52 [95% confidence interval 1.46-1.60]) and 27% for Part B (1.27 [1.18-1.38]). Each EPM point awarded for degrees increased the likelihood of passing Part A by 29% (1.29 [1.12-1.48]). SJT score was not a statistically significant independent predictor of MRCS success (P>0.05). Conclusions This is the first study to investigate the relationship between UK medical school performance and postgraduate surgical exam success. Higher ranked students are more likely to succeed in the MRCS compared to their lower ranked peers.