About the Study

In 2017, the Chief Evaluation Office (CEO) partnered with the Employment and Training Administration (ETA) and funded Mathematica Policy Research to conduct the Comparing Job Training Impact Estimates using Survey and Administrative Data study under the Administrative Data Research and Analysis portfolio of studies. The statistical analysis drew on three data sources measuring earnings and employment covering study participants in the WIA Gold Standard Evaluation: the WIA Gold Standard Evaluation surveys, the National Directory of New Hires (NDNH), and administrative tax data (analyzed separately by Manoli and Patel 2018). The analysis compared earnings levels and impact estimates based on all three sources to gather insights about the strengths, drawbacks, and tradeoffs inherent in pursuing each data source to answer research questions. Researchers compared data for 4,777 participants, and the study explores how and why the earnings impact estimates differ, though some aspects of the analysis only consist of 44% of this sample due to timing issues of the administration of the survey.

When researchers design evaluations of workforce development programs, they choose from data sources which all have benefits and drawbacks. Surveys of program participants can be granular but rely on respondents’ truthfulness and accuracy. Administrative datasets like the NDNH may be less expensive than surveys but require coordination between researchers and the agency in control of the data. Administrative tax records should capture all taxable income, but because they are collected annually, may not offer the timeliness that other options allow. Evaluation studies support programmatic and policy decisions, so the choice of data source in accurately measuring outcomes is important.

This Department of Labor-funded study contributes to the labor evidence-base to inform employment and training programs and policies and addresses Departmental strategic goals and priorities.

The Department of Labor’s (DOL) Chief Evaluation Office (CEO) sponsors independent evaluations and research, primarily conducted by external, third-party contractors in accordance with the Department of Labor Evaluation Policy and CEO’s research development process.