Effort is key in learning, evidenced by its omnipresence in both empirical findings and educational theories. At the same time, students are consistently found to avoid effort. In this study, we investigate whether limiting effort avoidance improves learning outcomes, and explore for whom this would be the case. In a large-scale computer adaptive practice system for primary education, over 150,000 participants were distributed across four conditions in which a problem-skipping option was delayed for 0, 3, 6, or 9 seconds. The results show that after a 14 week period, no average treatment effects in learning outcomes can be found between conditions. A substantive typology of students, based on the expected target mechanisms of the intervention, neither shows consistent conditional average treatment effects. Nevertheless, the substantive typology is shown to be meaningful, as the different types—toilers, skippers, and rushers—differ with respect to their learning outcomes. We argue that although the scale of the experiment suggests a precise null finding, the cumulative nature of the effect of problem skipping cautions against generalizing this finding to sustained intervention.
Estimation and Validation of Ratio-based Conditional Average Treatment Effects Using Observational Data
