Empowering Teachers with AI: Co-Designing a Learning Analytics Tool for Personalized Instruction in the Science Classroom

AI-based educational technology that is designed to support teachers in providing personalized instruction can enhance their ability to address the needs of individual students, hopefully leading to better learning gains. This paper presents results from participatory research aimed at co-designing with science teachers a learning analytics tool that will assist them in implementing a personalized pedagogy in blended learning contexts. The development process included three stages. In the first, we interviewed a group of teachers to identify where and how personalized instruction may be integrated into their teaching practices. This yielded a clustering-based personalization strategy. Next, we designed a mock-up of an AI-based tool that supports this strategy and worked with another group of teachers to define an `explainable learning analytics' scheme that explains each cluster in a way that is both pedagogically meaningful and can be generated automatically. Third, we developed an AI algorithm that supports this `explainable clusters' pedagogy and conducted a controlled experiment that evaluated its contribution to teachers' ability to plan personalized learning sequences. The planned sequences were evaluated in a blinded fashion by an expert, and the results demonstrated that the experimental group -- teachers who received the clusters with the explanations -- designed sequences that addressed the difficulties exhibited by different groups of students better than those designed by teachers who received the clusters without explanations. The main contribution of this study is twofold. First, it presents an effective personalization approach that fits blended learning in the science classroom, which combines a real-time clustering algorithm with an explainable-AI scheme that can automatically build pedagogically meaningful explanations from item-level meta-data (Q Matrix). Second, it demonstrates how such an end-to-end learning analytics solution can be built with teachers through a co-design process and highlights the types of knowledge that teachers add to system-provided analytics in order to apply them to their local context. As a practical contribution, this process informed the design of a new learning analytics tool that was integrated into a free online learning platform that is being used by more than 1000 science teachers.