The field of intelligent tutoring systems has successfully delivered techniques and applications to provide personalized coaching and feedback for problem solving in a variety of domains. The core of this personalized instruction is a student model; the ITS component in charge of assessing student traits and states relevant to tailor the tutorial interaction to specific student needs during problem solving. There are however, other educational activities that can help learners acquire the target skills and abilities at different stages of learning including, among others, exploring interactive simulations and playing educational games. This article describes research on creating student models that support personalization for these novel types of interactions, their unique challenges, and how AI and machine learning can help.