Improving out-of-population prediction: The complementary effects of model assistance and judgmental bootstrapping

Abstract

We propose and test a method for out-of-population prediction termed model-assisted judgmental bootstrapping, which leverages a predictive model from one domain combined with expert judgment to generate training data and subsequently a predictive model for a new domain. In a preregistered experiment (N=1440), we assessed the predictive accuracy of this method in increasingly challenging environments. We also analyzed the individual contributions of two techniques that underlie the method: model-assisted estimation and judgmental bootstrapping. Our findings revealed that both techniques significantly improved predictive accuracy. Furthermore, their impacts were complementary: model-assisted estimation provided the largest accuracy gains in the least demanding environment, while judgmental bootstrapping did so in the most challenging environment. Our results suggest that model-assisted judgmental bootstrapping is a promising technique for creating predictive models in domains in which outcome data are not available.