Score contribution per author:
α: calibrated so average coauthorship-adjusted count equals average raw count
A subset of undergraduate applicants to the University of California, Berkeley were invited to submit letters of recommendation as part of their applications. I use scraped text of the submitted letters, natural language processing tools, and a within-subject experimental design wherein applications were read in parallel with and without their letters to understand the role that this qualitative information plays in admissions. I show that letters written on behalf of underrepresented applicants were modestly distinctive. I also construct an index of letter strength, measuring the predicted impact of the letter on the student's application score. I show that underrepresented applicants tend to get weaker letters, but that readers pay less attention to letter strength for underrepresented students. Overall, the inclusion of letters modestly improved application outcomes for the average underrepresented student.