Shooting for the Stars

Persistence is considered a crucial factor for success in online learning environments. However, in interactive game-based learning environments, persistence in progressing in the game may come at the expense of investing in each of the game's levels. That is, the motivation to complete the game may have a deleterious effect on learning at specific levels and hence on learning from the game in general. Therefore, it is imperative that research focuses on micro-persistence, i.e., persistence during each component of the learning process. Taking a learning analytics approach, this large-scale log-based study (N=25,812 elementary- and middle-school students) examines micro-persistence within the context of learning computational thinking, a key skill for the 21st-century. Data was collected and analyzed from an online, game-based learning environment (CodeMonkey™). Results suggest that the acquisition of computational thinking is a multi-dimensional process, and that persistence is a crucial factor for success in multi-level game-based learning environments. The authors also found that game-based learning environments may prove effective in narrowing the gap between high-and low-achieving students.

Using Motivational Modelling With an App Designed to Increase Student Performance and Retention

Quitch is software designed to increase student performance and retention. It is a content-neutral, gamified mobile learning platform used across many disciplines, including accounting, chemistry, and engineering. The aim of Quitch is to ensure students feel engaged with their learning. Motivational modelling is a high-level approach to understand the purpose of a system. It is novel in its incorporation of emotional factors. This chapter discusses how the authors applied motivational modelling to Quitch to explain its purpose and potential. The chapter then more generally discusses how their modelling approach can help with the design and development of new software applications especially in the education space.

Use of Khan Academy and Mathematics Achievement

In summer 2017, Long Beach Unified School District partnered with Khan Academy to support pilot teachers in their implementation of Khan Academy in the classroom. This study examined how the use of Khan Academy in the middle school mathematics classroom relates to student achievement on the mathematics portion of the state standardized assessment. Results indicated that students who used Khan Academy for more than 30 minutes a week, the recommended usage time, scored an additional 22 points (0.20 standard deviation units) on the 2018 mathematics portion of the Smarter Balanced Assessment, compared to students who did not use Khan Academy. Additionally, these results hold true regardless of race/ethnicity, gender, eligibility for free/reduced lunch, or English learner status.

Capturing Student Affect

Early warning systems rely on behavioral and cognitive data drawn from student information systems, learning management systems, and courseware platforms; however, they often lack sufficient data on student attitudes, perceptions, and affective responses to effectively prevent student withdrawals, failures, and drop outs or to intervene early enough to improve institutional and student outcomes. To complement the behavioral and cognitive datastreams, researchers and designers are increasingly turning toward microsurveys—short questions or question sets that help researchers gather data at strategic points during a course—to enable earlier intervention and, therefore, improve outcomes. However, for microsurveys to be effective, researchers and designers may need to refactor their research, design, and evaluation processes to address considerations unique to microsurveys. This chapter considers how researchers may go about developing microsurveys by formulating a foundational research base, developing initial designs, and then refining those through formative evaluation.

Non-Cognitive Signals and Systems

Student success initiatives in practice, underlying technological infrastructure, and human processes, focus almost exclusively on cognitive signals for risk, persistence, and other alert factors. Yet decades of research suggest that these signals are quite limited because learners are not compartmentalized as cognitive beings vs affective beings vs conative beings. This chapter looks to inform the creation of both technological and human measurement as well as intervention techniques for a much more holistic approach to student success efforts as told through a case study of such a system. The chapter will help technologists, researchers, and service-practitioners alike in building workflows and technological systems to promote better inputs, better triggers, and better outputs, all for human consumption in the assistance of helping students thrive.

Dashboard for the E-Assessment and E-Feedback System for Aerospace Engineering Examination Preparation in Singapore

The e-Assessment and e-Feedback System (eAFS) provides trainees at the Temasek Polytechnic - Lufthansa Technical Training Centre with online assessments and feedback as overall scores. In 2017, personalized dashboard printouts were used to enhance feedback for the Materials and Hardware (M06) practice examination. Each presented a trainee's score for each topic vs the average score of the cohort, total scores of individuals and percentage of grades achieved by the tutorial class vs the highest, lowest, average and individual scores of the cohort. This study examined how participants responded to the personalized dashboard and if and how it influenced their final examination preparation. Data came from two surveys, a staff interview, and practice and actual examination scores. The participant's examination preparation strategies were somewhat influenced by data in their personalized dashboard. However, most reverted to the more standard strategies for their actual final examination preparation. These insights will influence how the trainee e-dashboard will be designed and used.

The Peril and Promise of Pre-Tests in Informal Massive Open Online Courses

With more than 100,000,000 learners from around the world, massive open online courses (MOOCs) are a popular online learning resource. Because this type of online teaching and learning is relatively young, published MOOC research is not as voluminous as traditional educational research. This presents both a challenge and an opportunity. The challenge is that best practices are not always clear, and there is not much MOOC research upon which to draw for specific instructional design strategies. The opportunity is to harness the power of MOOC platforms themselves to conduct research that examines and identifies effective digital pedagogy. In this chapter, the authors describe some of these challenges and opportunities. Specifically, they draw upon a multivariate experimental research study that examined the effects of pre-tests and feedback on learning and persistence in a MOOC. They offer practical implications that are related to study findings.

Do Student-Written Responses to Reflection Questions Predict Persistence and Performance in Online Courses?

Online learning has been recognized as a promising approach to improve learning outcomes in developing countries where high-quality learning resources are limited. Concomitant with the boom in online learning, there are escalating concerns about academic accountability, specifically student outcomes as measured by persistence and success. This chapter examines whether evidence of reflection found in student written responses to a series of skill-building videos predicts success in online courses. Using a text analysis approach, this study analyzed 1,871 student responses to four reflection questions at a large online university in Panama. A binary logistic regression analysis was conducted to explore whether student persistence was affected by evidence of words associated with significant learning found in student written responses to a set of reflection questions. The results suggest that evidence of words associated with significant learning found in student written responses to reflection questions significantly predicts student persistence in online courses. A Kruskal-Wallis test found median final course grade differences between students who showed no evidence of significant learning in their written responses, and those using 1-13 words associated with significant learning. These results strongly suggest that persistence and performance in online courses are affected by evidence of reflection found in student written responses to reflection questions. These results suggest that a set of reflection tasks assigned early in the course may prove effective in identifying at-risk students.

From Adaptive Learning Support to Fading Out Support for Effective Self-Regulated Online Learning

This chapter applies data mining and learning analytics, along with self-regulated learning (SRL) theories, to examine possible interventions aimed at supporting students' success with online learning. The chapter introduces two learning support systems and the results of related research. These two systems are used as sample cases to describe the relationships among SRL, learning support, learning processes, and learning effects. Case 1 is an early warning system that uses an SRL questionnaire completed before actual learning to determine which students are likely to drop out. Case 2 focuses on student planning and the implementation phases of the SRL cycle. This system supports students' own planning and learning, creating distributed learning and reducing procrastination without human intervention. A comparison of the two cases implies that a combination of an early warning system and system constraints that require planning before actual learning can reduce the need for human learning support and decrease academic procrastination, resulting in increased distributed learning.

The Impact of a Cohort Model for Online Doctoral Student Retention and Success

Two critical decisions were made in the design of Canada's first fully online doctoral program discussed in this chapter: to create a professional Doctorate in Education rather than a PhD and to enroll students as cohorts each year. The first decision was based on the contemporary need within the field of online higher education for discipline specialists to have a solid background in online education principles and practice. The second decision was made on the basis of literature around benefits for graduate students. However, little sustained research has been carried out on what specific benefits may accrue for doctoral students participating in a cohort-based program in an online environment. This chapter presents and discusses the outcomes of two research studies on a cohort model, to provide insights into some of the personal and other factors identified as early warning indicators of student difficulties and how and when they arise.