Analysis of engagement metrics vs student attainment using multiple sources across physical and virtual spaces: A case study.

Learning analytics, referring to the collection and analysis of data regarding the progress of learners, allows higher education institutions and individual academics to make data driven decisions regarding their teaching approaches and support they are providing. Further, they provide students with an opportunity to take control of their own learning, as they are gaining a better understanding of their own performance and can make informed decisions about their own learning progress. In early 2020 a global pandemic forced higher education institutions worldwide to quickly move teaching online. We argue that under these circumstances, detailed learning analytics provide a unique opportunity to understand student behaviour and support individual learning. We present a case study analysing engagement metrics and their relationship to student attainment in four courses in the area of behavioural biology, over a time period of two years pre-pandemic. Multiple sources of student engagement in the physical (attendance at lectures) and virtual space (access and engagement with online learning resources) were used. Our results show that grades of students were significantly affected by type of assignment, with grades being lower in exams compared to other types of assignment. Grades were not significantly affected by level of studies, gender and country of origin (UK versus non-UK). With regards to engagement metrics, grades significantly increased with percentage of attendance in class, percentage of resources accessed on Canvas and library access. Students accessed lecture notes longer compared to other resources. Physical attendance in class over all courses and levels of studies averaged at 55 %. Online, students accessed on average only 32 % of resources provided in the virtual learning environment. Students accessed the majority of the courses in the same week when materials were discussed in class compared to the weeks before and after. Our results show that both engagement with materials in the virtual learning environment and attendance in class are positively correlated with student achievement. We cannot make any inferences about the causality of this effect and it is likely that better students in general are more engaged. Our project provides detailed in-depth insight into student behaviour and reveals that students overall do not engage with all materials provided, resulting in an incomplete learning experience. We suggest that detailed data on engagement of students with individual resources can be used to better understand and shape individual learning experiences of students.