The Fragile Families Challenge: A Scientific Mass Collaboration
Sociologists have long theorized about the processes through which childhood experiences shape life outcomes. However, statistical models that use data on family background and childhood experiences to predict life outcomes often have poor predictive performance. These empirical results have lead pioneers of the field muse that random chance must play an important role. In this talk, we present results from the Fragile Families Challenge, a scientific mass collaboration based on the Fragile Families and Child Wellbeing Study, a birth cohort study of about 5,000 families from large US cities. The study began with a probability sample of newborns, and for more than 15 years, researchers have followed these families to collect information related to child and family development as reported by the child as well as the child’s mother, father, primary caregiver, and teachers. During the Fragile Families Challenge, 159 research teams from 68 institutions in 7 countries used this high-dimensional survey data to produce statistical and machine learning models to predict six life outcomes. Results suggest that (a) modern machine learning methods enabled predictive performance that outpaced approaches more common in social science, but (b) overall predictive performance was poor. The talk will include a discussion of the potential reasons for poor predictive performance in this setting, open methodological questions raised by our results, and the potential for mass collaboration to address other scientific and policy questions.
Matthew Salganik is Professor of Sociology at Princeton University, and he is affiliated with several of Princeton's interdisciplinary research centers: the Office for Population Research, the Center for Information Technology Policy, the Center for Health and Wellbeing, and the Center for Statistics and Machine Learning. His research interests include social networks and computational social science. He is the author of Bit by Bit: Social Research in the Digital Age.
Salganik's research has been published in journals such as Science, PNAS, Sociological Methodology, and Journal of the American Statistical Association. His papers have won the Outstanding Article Award from the Mathematical Sociology Section of the American Sociological Association and the Outstanding Statistical Application Award from the American Statistical Association. Popular accounts of his work have appeared in the New York Times, Wall Street Journal, Economist, and New Yorker. Salganik's research is funded by the National Science Foundation, National Institutes of Health, Joint United Nations Program for HIV/AIDS (UNAIDS), Russell Sage Foundation, Sloan Foundation, Facebook, and Google. During sabbaticals from Princeton, he has been a Visiting Professor at Cornell Tech and a Senior Research at Microsoft Research.