On the generalizability of using mobile devices to conduct economic experiments

Abstract

Mobile devices enable experimental economists to collect decision-making data from more heterogeneous samples, thereby increasing the generalizability of their results. This generalizability may be compromised if the device is a relevant behavioural confound. This paper reports on a battery of classic economic games and tasks in which we randomize the decision-making device (computer versus mobile) and the laboratory setup (physical versus remote). Our results offer broad support for conducting decision experiments using mobile devices. For six out of eight tasks, we find robust null results for means of treatment effects as well for differences in variances and general distributions of choices across treatments. This should give researchers confidence to conduct studies via mobile devices and in out-of-lab settings. However, we find two caveats. First, subjects using a mobile device exhibit more risk aversion and offer less during bargaining. These effects persist in the physical lab after controlling for subjects’ observed characteristics. Second, response times and the time taken to read instructions using a mobile device are shorter in the remote setup. These qualifications suggest the importance of ensuring device consistency across treatments.