The effect of adaptive versus static practicing on student learning - evidence from a randomized field experiment

Abstract

Schools and governments are increasingly investing in adaptive practice software. To date, the evidence whether adaptivity improves learning outcomes is limited and mixed. A large-scale randomized control trial is conducted in Dutch secondary schools to evaluate the effectiveness of an adaptive practice program relative to a static program. Learning theories predict that adaptive practicing is more effective, but this experimental evaluation provides a more nuanced picture. Relative to the static software environment, students working in the adaptive software environment receive more difficult exercises, practice longer and answer fewer questions correctly. Takeup and usage of the software program is, overall, modest, but varies considerably within and between classrooms. The outcome differences between both environments are more pronounced in classrooms with higher practice intensity. On average, no test score effects are found, but static practicing does improve test scores for higher ability students (0.08σ). Caution is thus warranted when adaptive practice software is implemented to address individual learning needs, as static formative test preparation can be more effective in improving test scores.