Library of Early Indicators
Call for Contributions to a Library of Early Indicators for Social Science
We invite interested researchers to contribute to a library of early indicators to predict the long-term impacts of social science interventions, drawing on data from previous experiments you may have analyzed. Specifically, we invite you to either (1) identify early indicators in your own data using a method we’ve recently developed (described below) or (2) collaborate with us to estimate the long-term impacts of interventions you have studied by linking individual-level data from your intervention to administrative panel data on earnings. We may be able to offer research assistance in both cases.
The Problem: A major challenge in estimating interventions’ impacts in social science is that there are long delays in observing long-term outcomes of interest (e.g., impacts of childhood interventions on lifetime earnings). Our aim is to overcome this challenge by building a library of early indicators to predict a variety of long-term outcomes, ranging from earnings to mortality to educational outcomes.
Our Approach: Our approach is motivated by our paper on surrogate indices (Athey, Chetty, Imbens, and Kang 2019), who developed a simple method of combining early indicators into “surrogate indices” – predicted values of long-term outcomes given early indicators — to predict long-term treatment effects. That paper presents an empirical application showing that the long-term (nine-year) impacts of a job-training program in California can be accurately estimated within 1.5 years using short-term employment indicators as surrogates. A short summary of the study is attached.
We aim to expand this analysis by identifying surrogate indices that perform well in predicting long-term impacts in a variety of other randomized experiments that researchers have conducted. Our team at Opportunity Insights will then collect and synthesize information about which sets of surrogate outcomes predict long-term impacts on a publicly available website. Our hope is that future researchers and policymakers will then be able to refer to this database of early indicators for policy evaluation, expediting the research cycle.
How You Can Contribute: There are two paths through which you can contribute to the library. The first path is relevant if you have previously estimated both short-term and long-term outcomes in an RCT. In this case, you can contribute to the public library by estimating a surrogate index that could be used to predict long-run outcomes, given the short-run outcomes you observed (as in Athey et al. 2019). We can offer research assistance to facilitate estimating this surrogate index.
The second path is to construct new estimates of the long-term impacts of interventions you have previously studied by linking data from your RCT to administrative data our team has constructed covering the U.S. population at the individual level. This path is relevant if you have previously studied short-term impacts in an RCT, and believe that long-term impacts could be identified using administrative data on earnings.
The success of this effort hinges on having a large set of studies to draw from, so we very much hope you will consider participating — both to create a public good and as a vehicle to amplify the impact of your own work. If you would like to participate in this effort or have any questions, please email:
Emanuel Schertz
Jimmy Stratton