Development and use of a computerized system to track the competency development of family medicine residents: analysis of the convergence between system proposals and assessor decisions

In recent decades, a number of training environments have moved toward program approaches targeting the development of competencies. Because of their complexity, monitoring the development of those competencies is a considerable challenge. Our hypothesis is that a computerized system could help overcome this challenge if it is well accepted by its users. We first summarize the context surrounding the implementation of such approaches. Next, we present a computerized assessment system established in the Family Medicine Residency Program of Laval University (Québec, Canada) that we have developed for tracking the development of residents’ competencies. We then present the analysis of interactions between the system and users and the various proposals that were made to improve the system and longitudinal tracking of the development of the targeted competencies. We consider that this research provides useful guidelines for the computerized monitoring of learners' competencies development and for the design of such systems.

Online student support: a framework for embedding support interventions into the online learning cycle

Support is one of the crucial elements of online students’ success. Although many support strategies have been documented in the past, less is known at what stages of the learning cycle suggested interventions can be embedded into the online learning curriculum. This paper offers a systematic review of the 28 empirical studies on effective support strategies and interventions that are indexed by the SCOPUS database between 2010 and 2020. Following an Inclusive Student Services Process Model framework, identified strategies are allocated across different phases of student learning to indicate where and when they can be delivered to online students. The analysis suggests that the effectiveness of the support provision depends on the time when support is offered. Furthermore, it was found that two areas support delivery, namely support at transitions and measurement of support interventions, remain under-researched. Finally, the analysis showed two emerging trends in online students support: an increasing role of technology and social network sites to design support interventions and a shift to a more personalised yet holistic approach to student support.

Robot lecture for enhancing presentation in lecture

In lectures with presentation slides such as an e-learning lecture on video, it is important for lecturers to control their non-verbal behavior involving gaze, gesture, and paralanguage. However, it is not so easy even for well-experienced lecturers to properly use non-verbal behavior in their lecture to promote learners’ understanding. This paper proposes robot lecture, in which a robot substitutes for human lecturers, and reconstructs their non-verbal behavior to enhance their lecture. Towards such reconstruction, we have designed a model of non-verbal behavior in lecture. This paper also demonstrates a robot lecture system that appropriately reproduces non-verbal behavior of human lecturers with reconstructed one. In addition, this paper reports a case study involving 36 participants with the system, whose purpose was to ascertain whether robot lecture with reconstruction could be more effective for controlling learners' attention and more beneficial for understanding the lecture contents than video lecture by human and robot lecture with simple reproduction. The results of the case study with the system suggest the effect of promoting learners’ understanding of lecture contents, the necessity of reconstructing non-verbal behavior, and the validity of the non-verbal behavior model.

Delivery and evaluation of an e-Learning framework through computer-aided analysis of learners’ reflection text in a teacher development course

This study aimed at proposing an e-Learning framework in school education. The proposed framework consisted of technology and pedagogy dimensions. The method of computer-aided analysis of learners’ reflection text was used to evaluate the pedagogical delivery of the proposed framework. A study involving 33 in-service teachers in a teacher development course was conducted to investigate the effectiveness of the pedagogical delivery. The computer-aided analysis of learners’ reflection text on “what is e-Learning” before and after the teaching of the course showed that learners deepened their understanding about the technology use and the importance of pedagogy in e-Learning, to consider more the component on theories/models/principles/strategies in pedagogical design and practices, emphasize more the reflection and discussion in learning and teaching activities, and also realize more the possibility of pedagogical decision-making using data collected from online learning. The questionnaire survey and focus group interview with learners also indicated that the pedagogical delivery of the e-Learning framework with the support from the computer-aided analysis of learners’ reflection text coupled with hierarchical visualization of analysis results was positively perceived. The research findings contribute pioneering insights into the use of a computer-aided approach for an accurate and efficient evaluation in teacher development courses on e-Learning.

Learning analytics of humanities course: reader profiles in critical reading activity

This study investigates learner’s reading behaviors in a critical reading task in humanities course using learning analytics techniques. A Critical Analysis of Literature and Cinema course was selected as a context. The course activities evolved over 10 years, and for this instance, some face-to-face classroom critical reading activities were migrated to online mode by using BookRoll, a learning analytics enhanced eBook platform. Students (n=22 out of the 50 registered) accessed Hayavadana, an Indian play uploaded on BookRoll, and attempted to identify performative elements and cultural references in the text and highlight them. In this study, we analyze learner’s reading logs gathered in the learning record store linked to BookRoll during that activity. We extend our previous work where we identify four online reading profiles: effortful, strategic, wanderers, and check-out, based on learner’s clickstream interactions and time spent with the content. We validate the profiles with qualitative interview data collected from the learners and illustrate the quantified learning behaviors of each of those profiles based on an engagement metric. Our work aims to initiate further discussion related to the application of learning analytics in humanities courses both to probe into the learning behaviors of the students and thereby enhance the experiences with the use of interactive learning environments and data-driven services.

Contract cheating: an increasing challenge for global academic community arising from COVID-19

Due to COVID-19, universities with limited expertise with the digital environment had to rapidly transition to online teaching and assessment. This transition did not create a new problem but has offered more opportunities for contract cheating and diversified the types of such services. While universities and lecturers were adjusting to the new teaching styles and developing new assessment methods, opportunistic contract cheating providers have been offering $50 COVID-19 discounts and students securing the services of commercial online tutors to take their online exams or to take advantage of real-time assistance from ‘pros’ while sitting examinations. The article contributes to the discourse on contract cheating by reporting on an investigation of the scope and scale of the growing problems related to academic integrity exacerbated by an urgent transition to online assessments during the COVID-19 pandemic. The dark reality is the illegal services are developing at a faster pace than the systems required to curb them, as demonstrated by the results. The all-penetrating issues indicate systemic failures on a global scale that cannot be addressed by an individual academic or university acting alone. Multi-level solutions including academics, universities and the global community are essential. Future research must focus on developing a model of collaboration to address this problem on several levels, taking into account (1) individual academics, (2) universities, (3) countries and (4) international communities.

A framework to foster analysis skill for self-directed activities in data-rich environment

Self-direction skill is considered a vital skill for twenty-first-century learners in both the learning context and physical activity context. Analysis skill for self-directed activities requires the students to analyze their own activity data for understanding their status in that activity. It is an important phase that determines whether an appropriate plan can be set or not. This research presents a framework designed to foster students’ analysis skill in self-directed activities, which aims (1) to build a technology-enabled learning system allowing students to practice analyzing data from their own daily contexts, (2) to propose an approach to model student’s analysis skill acquisition level and process, and (3) to provide automated support and feedback for analysis skill development tasks. The analysis module based on the proposed framework was implemented in the GOAL system which synchronized data from learners’ physical and reading activities. A study was conducted with 51 undergraduate students to find reliable indicators for the model to then measure students’ analysis skills. By further analyzing students’ actual usage of the GOAL system, we found the actual activity levels and their preferences regarding analysis varied for the chosen contexts (learning and physical activity). The different context preference groups were almost equal, highlighting the utility of a system that integrates data from multiple contexts. Such a system can potentially respond to students’ individual preferences to execute and acquire self-direction skill.