Measuring the predictability of life outcomes with a scientific mass collaboration

Matthew J. Salganik; Ian Lundberg; Alexander T. Kindel; Caitlin E. Ahearn; Khaled Al-Ghoneim; Abdullah Almaaatouq; Drew M. Altschul; Jennie E. Brand; Nicole Bohme Carnegie; Ryan James Compton; Debanjan Datta; Thomas Davidson; Anna Filippova; Connor Gilroy; Brian J. Goode; Eaman Jahani; Ridhi Kashyap; Antje Kirchner; Stephen McKay; A Morgan; Alex Pentland; Kivan Polimis; Louis Raes; Daniel E. Rigobon; Claudia V. Roberts; Diana Stanescu; Yoshihiko Suhara; Adaner Usmani; Erik Wang

Measuring the predictability of life outcomes with a scientific mass collaboration

Matthew J. Salganik, Ian Lundberg, Alexander T. Kindel, Caitlin E. Ahearn, Khaled Al-Ghoneim, Abdullah Almaaatouq, Drew M. Altschul, Jennie E. Brand, Nicole Bohme Carnegie, Ryan James Compton, Debanjan Datta, Thomas Davidson, Anna Filippova, Connor Gilroy, Brian J. Goode, Eaman Jahani, Ridhi Kashyap, Antje Kirchner, Stephen McKay, A MorganAlex Pentland, Kivan Polimis, Louis Raes, Daniel E. Rigobon, Claudia V. Roberts, Diana Stanescu, Yoshihiko Suhara, Adaner Usmani, Erik Wang

Research output: Contribution to journal › Article

Abstract

How predictable are life trajectories? We investigated this question with a scientific mass collaboration using the common task method; 160 teams built predictive models for six life out comes using data from the Fragile Families and Child Wellbeing Study, a high-quality birth cohort study. Despite using a rich dataset and applying machine-learning methods optimized for prediction, the best predictions were not very accurate and were only slightly better than those from a simple benchmark model. Within each outcome, prediction error was strongly associated with the family being predicted and weakly associated with the technique used to generate the prediction. Overall, these results suggest practical limits to the predictability of life outcomes in some settings and illustrate the value of mass collaborations in the social sciences.

Original language	English
Pages (from-to)	8398-8403
Journal	PNAS - Proceedings of the National Academy of Sciences of the United States of America
Volume	117
Issue number	15
Publication status	Published - 2020

Cite this

Salganik, M. J., Lundberg, I., Kindel, A. T., Ahearn, C. E., Al-Ghoneim, K., Almaaatouq, A., Altschul, D. M., Brand, J. E., Carnegie, N. B., Compton, R. J., Datta, D., Davidson, T., Filippova, A., Gilroy, C., Goode, B. J., Jahani, E., Kashyap, R., Kirchner, A., McKay, S., ... Wang, E. (2020). Measuring the predictability of life outcomes with a scientific mass collaboration. PNAS - Proceedings of the National Academy of Sciences of the United States of America, 117(15), 8398-8403.

@article{3072bb95553a42f1bae1996dbd7e7efe,

title = "Measuring the predictability of life outcomes with a scientific mass collaboration",

abstract = "How predictable are life trajectories? We investigated this question with a scientific mass collaboration using the common task method; 160 teams built predictive models for six life out comes using data from the Fragile Families and Child Wellbeing Study, a high-quality birth cohort study. Despite using a rich dataset and applying machine-learning methods optimized for prediction, the best predictions were not very accurate and were only slightly better than those from a simple benchmark model. Within each outcome, prediction error was strongly associated with the family being predicted and weakly associated with the technique used to generate the prediction. Overall, these results suggest practical limits to the predictability of life outcomes in some settings and illustrate the value of mass collaborations in the social sciences.",

author = "Salganik, {Matthew J.} and Ian Lundberg and Kindel, {Alexander T.} and Ahearn, {Caitlin E.} and Khaled Al-Ghoneim and Abdullah Almaaatouq and Altschul, {Drew M.} and Brand, {Jennie E.} and Carnegie, {Nicole Bohme} and Compton, {Ryan James} and Debanjan Datta and Thomas Davidson and Anna Filippova and Connor Gilroy and Goode, {Brian J.} and Eaman Jahani and Ridhi Kashyap and Antje Kirchner and Stephen McKay and A Morgan and Alex Pentland and Kivan Polimis and Louis Raes and Rigobon, {Daniel E.} and Roberts, {Claudia V.} and Diana Stanescu and Yoshihiko Suhara and Adaner Usmani and Erik Wang",

year = "2020",

language = "English",

volume = "117",

pages = "8398--8403",

journal = "PNAS - Proceedings of the National Academy of Sciences of the United States of America",

number = "15",

}

Salganik, MJ, Lundberg, I, Kindel, AT, Ahearn, CE, Al-Ghoneim, K, Almaaatouq, A, Altschul, DM, Brand, JE, Carnegie, NB, Compton, RJ, Datta, D, Davidson, T, Filippova, A, Gilroy, C, Goode, BJ, Jahani, E, Kashyap, R, Kirchner, A, McKay, S, Morgan, A, Pentland, A, Polimis, K, Raes, L, Rigobon, DE, Roberts, CV, Stanescu, D, Suhara, Y, Usmani, A & Wang, E 2020, 'Measuring the predictability of life outcomes with a scientific mass collaboration', PNAS - Proceedings of the National Academy of Sciences of the United States of America, vol. 117, no. 15, pp. 8398-8403.

TY - JOUR

T1 - Measuring the predictability of life outcomes with a scientific mass collaboration

AU - Salganik, Matthew J.

AU - Lundberg, Ian

AU - Kindel, Alexander T.

AU - Ahearn, Caitlin E.

AU - Al-Ghoneim, Khaled

AU - Almaaatouq, Abdullah

AU - Altschul, Drew M.

AU - Brand, Jennie E.

AU - Carnegie, Nicole Bohme

AU - Compton, Ryan James

AU - Datta, Debanjan

AU - Davidson, Thomas

AU - Filippova, Anna

AU - Gilroy, Connor

AU - Goode, Brian J.

AU - Jahani, Eaman

AU - Kashyap, Ridhi

AU - Kirchner, Antje

AU - McKay, Stephen

AU - Morgan, A

AU - Pentland, Alex

AU - Polimis, Kivan

AU - Raes, Louis

AU - Rigobon, Daniel E.

AU - Roberts, Claudia V.

AU - Stanescu, Diana

AU - Suhara, Yoshihiko

AU - Usmani, Adaner

AU - Wang, Erik

PY - 2020

Y1 - 2020

N2 - How predictable are life trajectories? We investigated this question with a scientific mass collaboration using the common task method; 160 teams built predictive models for six life out comes using data from the Fragile Families and Child Wellbeing Study, a high-quality birth cohort study. Despite using a rich dataset and applying machine-learning methods optimized for prediction, the best predictions were not very accurate and were only slightly better than those from a simple benchmark model. Within each outcome, prediction error was strongly associated with the family being predicted and weakly associated with the technique used to generate the prediction. Overall, these results suggest practical limits to the predictability of life outcomes in some settings and illustrate the value of mass collaborations in the social sciences.

AB - How predictable are life trajectories? We investigated this question with a scientific mass collaboration using the common task method; 160 teams built predictive models for six life out comes using data from the Fragile Families and Child Wellbeing Study, a high-quality birth cohort study. Despite using a rich dataset and applying machine-learning methods optimized for prediction, the best predictions were not very accurate and were only slightly better than those from a simple benchmark model. Within each outcome, prediction error was strongly associated with the family being predicted and weakly associated with the technique used to generate the prediction. Overall, these results suggest practical limits to the predictability of life outcomes in some settings and illustrate the value of mass collaborations in the social sciences.

M3 - Article

VL - 117

SP - 8398

EP - 8403

JO - PNAS - Proceedings of the National Academy of Sciences of the United States of America

JF - PNAS - Proceedings of the National Academy of Sciences of the United States of America

IS - 15

ER -

Measuring the predictability of life outcomes with a scientific mass collaboration

Abstract

Fingerprint

Cite this