Beyond RoboDebt

Automated decision support systems with high stake decision processes are frequently controversial. The Online Compliance Intervention (herewith “OCI” or “RoboDebt”) is a system of compliance implemented with the intention to facilitate automatic issuance of statutory debt notices to individuals, taking a receipt of welfare payments and exceeding their entitlement. The system appears to employ rudimentary data scraping and expert systems to determine whether notices should be validly issued. However, many individuals that take receipt of debt notices assert that they were issued in error. The commentary on the system has resulted in a lot of conflation of the system with other system types and caused many to question the role of decision of support systems in public administration given the potentially deleterious impacts of such systems for the most vulnerable. The authors employ a taxonomy of Robotic Process Automation (RPA) issues, to review the OCI and RPA more generally. This paper identifies potential problems of bias, inconsistency, procedural fairness, and overall systematic error. This research also considers a series of RoboDebt specific issues regarding contractor arrangements and the potential impact of the system for Australia's Indigenous population. The authors offer a set of recommendations based on the observed challenges, emphasizing the importance of moderation, independent algorithmic audits, and ongoing reviews. Most notably, this paper emphasizes the need for greater transparency and a broadening of criteria to determine vulnerability that encompasses, temporal, geographic, and technological considerations.

Burnout Among Healthcare Professionals in Bosnia and Herzegovina

Burnout, a state of stress-induced emotional, physical, and mental exhaustion, continues to be a topic of interest across a broad array of sciences. It is because burnout not only causes psychosomatic problems, but also has an effect on job performance, which is vital in high-stake professions. Therefore, exploring the level of healthcare professionals’ burnout, and understanding which work-place factors are correlated with it, is of outmost importance. For this purpose, the present correlational study explored this issue in a convenience sample of 209 MDs and nurses from primary healthcare institutions in Bosnia and Herzegovina, including, for the first time, a comprehensive number of psychosocial factors at work. Interestingly, the majority of healthcare professionals scored low on burnout measures of MBI. Nevertheless, one fifth of participants had potential early warning signs of burnout. Compared to their colleagues in other European countries, Bosnian-Herzegovinian doctors experience lower yet comparable levels of emotional exhaustion, lower depersonalization, and higher sense of personal achievement. Furthermore, nurses show an even more beneficial trend on all three burnout dimensions. In addition, the study established some significant positive predictors of burnout dimensions pertaining to the work environment. More precisely, quantitative workload and decision-making demands were found to be positive predictors of emotional exhaustion, while the strongest predictors of depersonalization were work-place support (from colleagues and superiors) and self-esteem. Significant predictors of personal accomplishment were perception of mastery and work centrality. This suggests that burnout among healthcare professionals arises both from the immediate workplace factors and individual ones, therefore implicating institutions in its prevention and reduction.

Delivering a modified continuous objective structured clinical examination for ophthalmology residents through a hybrid online method

Since coronavirus disease 2019 was declared a global pandemic by the World Health Organization, it has become a challenging situation to continue medical education, including in Indonesia. The situation prohibited face-to-face (direct) educational activities in clinical settings, therefore also postponing examinations involving especially procedural skills. Adaptations were urgently needed to maintain the delivery of high-stake examinations to sustain the number of ophthalmology graduates and the continuation of eye health service. Objective structured clinical examination (OSCE) has been one of our widely used method to assess clinical competencies for ophthalmology residents, and is the one method that involves gatherings, close contact of examiners, examinees and patients, therefore the most difficult to adjust. Pandemic challenges brought technical changes in our delivering the OSCE to online, maximizing digital platforms of meetings, while still concerned to guarding the safety of candidates, patients and staffs. OSCE scenarios were also made as timely efficient as possible by changing continuous station models to a cascade one. The purpose of this article is to document our experience in conducting a feasible and reproducible OSCE in this pandemic era filled with limitations.

Does Managerial Use of Performance Information Matter to Organizational Outcomes?

Public management scholars have made impressive strides in explaining managerial usage of performance information (PI). Does such PI use matter to performance? If so, what types of use make a difference? To answer these questions, we connect managerial self-reported behavior with objective organizational outcomes in Texas schools. We control for lagged comparative school performance and employ inverse probability weighting to mitigate endogeneity concerns. The results show that managerial use of PI is associated with objective indicators of performance, and that the type of use matters. In particular, school principals’ use of PI for strategic planning is positively associated with better high-stake test scores. The findings suggest that maturity of performance management system can shape the relationship between managerial PI use and organizational performance, thereby contributing to a contingency-based understanding of the relationship between performance management and organizational performance.

Project portfolio risk analysis with the consideration of project interdependencies

PurposeComprehensive project portfolio risk (PPR) analysis is essential for the success and sustainable development of project portfolios (PPs). However, project interdependency creates complexity for PPR analysis. In this study, considering the interdependency effect among projects, the authors develop a quantitative evaluation model to analyze PPR based on a fuzzy Bayesian network.Design/methodology/approachIn this paper, the primary purpose is to comprehensively evaluate project portfolio risk considering the interdependency effect using a systematical model. Accordingly, a fuzzy Bayesian network (FBN) is developed based on the existing studies. Specifically, first, the risks in project portfolios are identified from the project interdependencies perspective. Second, a fuzzy Bayesian network is adopted to model and quantify the interaction relationships among risks. Finally, the model is implemented to analyze the occurrence situation and characteristics of risks.FindingsThe interdependency effect can lead to high-stake risks, including weak financial liquidity, a lack of cross-project members and project priority imbalance. Furthermore, project schedule risks and inconsistency between product supply and market demand are relatively sensitive and should also be prioritized. Also, the validity of this risk evaluation model has been proved.Originality/valueThe findings identify the most sensitive risks for guaranteeing portfolio implementation and reveal interdependency effect can trigger some specific risks more often. This study proposes for the first time to measure and analyze project portfolio risk by a systematical model. It can help systematically assess and manage the complicated and interdependent risks associated with project portfolios.

Educators and practitioners’ perspectives in the development of a learning by concordance tool for medical clerkship in the context of the COVID pandemic

Background: The COVID-19 pandemic has forced medical schools to create educational material to palliate the anticipated and observed decrease in clinical experiences during clerkships. An online learning by concordance (LbC) tool was developed to overcome the limitation of students’ experiences with clinical cases. However, knowledge about the instructional design of an LbC tool is scarce, especially the perspectives of collaborators involved in its design: 1- educators who wrote the vignettes’ questions and 2- practitioners who constitute the reference panel by answering the LbC questions. The aim of this study was to describe the key elements that supported the pedagogical design of an LbC tool from the perspectives of educators and practitioners. Methods: A descriptive qualitative research design has been used. Online questionnaires were used, and descriptive analysis was conducted. Results: Six educators and 19 practitioners participated in the study. Important to the educators in designing the LbC tool were prevalent or high-stake situations, theoretical knowledge, professional situations experienced and perceived difficulties among students, and that the previous workshop promoted peer discussion and helped solidify the writing process. Important for practitioners was standards of practice and consensus among experts. However, they were uncertain of the educational value of their feedback, considering the ambiguity of the situations included in the LbC tool. Conclusions: The LbC tool is a relatively new training tool in medical education. Further research is needed to refine our understanding of the design of such a tool and ensure its content validity to meet the pedagogical objectives of the clerkship.

The relationship between topic and metaphor in second-language learners’ essays

In this article I investigate to what extent the use of metaphorical expressions in language learners’ texts vary according to the topic they have chosen to write about. The data come from the Norwegian learner corpus ASK, where the texts are from written assignments produced by adult second-language learners as part of an official Norwegian test and texts. Texts from two different prompts are selected, which are related to friendship and nature. Metaphors are defined according to conceptual metaphor theory and a triangulation of methods is used, alternating between a manual and an automatic extraction method. The results confirm the hypothesis that the two different prompts given to the learners in a language test not only triggers different metaphorical expressions but also influences the amount of metaphor used in the learners’ writing. This knowledge is important to researchers for comparing the use of metaphors between different groups, such as between different learners or between students in different stages of education. It is also important for test designers who decide on topics to be used in tests and teachers who help learners prepare for their tests. In addition, it is of interest for researchers, educators in general and the learners themselves who are interested in the effect the use of metaphors in texts have on raters’ evaluations in high-stake tests.

The use of a formative OSCE to prepare emergency medicine residents for summative OSCEs: a mixed-methods cohort study

Background The objective structured clinical examination (OSCE) is a part of emergency medicine (EM) examinations such as the Masters of Medicine in Emergency Medicine (MMed) examination and the equivalent Member of the Royal College of Emergency Medicine (MRCEM) examination. The use of formative OSCEs to prepare EM residents for summative OSCEs has not been investigated. This study aimed to evaluate the role of formative OSCEs in preparing EM residents for the MMed and MRCEM OSCE. Methods This was an observational, retrospective, mixed-methods cohort study. We analysed data from formative OSCEs conducted by the National Healthcare Group EM residency programme from 2013 to 2019, and from a questionnaire distributed to all residents during the study period. Residents’ formative OSCE participation and scores were compared with first-attempt summative OSCE success. Qualitative analysis of residents’ opinions on the usefulness of the formative OSCE was performed. Results Forty-three of the 50 (86.0%) residents attended at least one formative OSCE. Of the 46 who responded to the questionnaire, 40 (87.0%) had attempted and succeeded in the MMed or MRCEM OSCE, of whom 35 (87.5%) had succeeded on the first attempt. Residents who succeeded in the summative OSCE on the first attempt tended to have higher proximate (mean = 70.6, SD = 8.9 vs mean = 64.3, SD = 10.8) and mean (mean = 67.4, SD = 7.1 vs mean = 62.8, SD = 7.3) formative OSCE scores. All 8/40 (20.0%) residents who attended more than three formative OSCEs succeeded in the summative OSCE on their first attempt. Residents’ formative OSCE scores tended to improve with successive formative OSCEs, demonstrating a positive training effect. All residents felt that the formative OSCE was useful in preparing them for the summative OSCE. Conclusions Participation in multiple formative OSCEs was beneficial in preparing residents for the summative OSCE. The formative OSCE was useful in familiarising residents with the examination, giving them an opportunity to perform in near-examination conditions, and providing feedback to residents and faculty about their progress. Our findings may support the implementation of formative OSCEs in other training programmes to prepare learners for high-stake summative OSCEs.

A Critique on Discourse of Language Tests

In the 21st century, with the rise of the popularity of standardized or large-scale tests, their high-stakes have started to be apparent. High-stake tests are not new, but in most cases, their current use as social practice tends to shape individuals’ futures. Currently the new trend for their quality discussion aims to critically evaluate tests through the focus on their functions, use and power in their testing discourse, whereas traditionally what was included in this discussion was only their psychometric features. Regarding those tests as social practices, examining the functions, consequences and use of tests in their own discourses is at the heart of this new perspective. Driven by the tenet of such a critical perspective, this study aims to first provide a better understanding of ‘discourse of test’, and then describe the social dimensions comprising discourse of language tests. Finally, this study concludes with some suggestions for adapting a critical perspective to improve the discourse of tests and enhance their quality.

Motivation in teacher education: The forgotten element and its snowball effect

This research explores student teachers' motivation to join teacher education and its relation to their career intention to join the teaching profession. The qualitative study was conducted in the English department of private and state teacher education in Indonesia. A total of 14 student teachers (20-23 years old) took part in the interview. The findings show that most student teachers chose Initial Teacher Education (ITE) because of external motivation factors such as the alternative option of not being accepted at their desired faculty or following parents’ suggestions. Besides that, considering the internal motivation factor, one of the common reasons students chose English teacher education was the interest in English, which was also viewed as transferrable skills that could be useful to many other job possibilities. The findings show that only a few of the participants who prioritised their career to become teachers. The findings might contest the generalisation that often associates student teachers’ motivation to join teacher education to become teachers. Even though the assumption is valid to some extent, the nuances in the student teachers’ motivation to choose teacher education might need to be considered, especially in the context when joining teacher education is relatively easy because there is no high-stake test involved and teaching might be perceived as an unattractive career intention. The findings also indicate that as motivation is fluid, abstract, and hard to measure, teacher education overlooks this element in the admission process. However, forgetting the importance of student teachers’ motivation in teacher education could lead to a more severe problem in the quality of graduates and education in the long run.