About the Report

Download Report

In 2017, the Chief Evaluation Office (CEO) partnered with the Employment and Training Administration (ETA) to fund contractors Urban Institute, George Washington University, Capital Research Corporation, and the National Association of State Workforce Agencies to conduct an analysis of employer performance measurement approaches required by Workforce Innovation and Opportunity Act (WIOA).

The report summarizes results and presents a set of considerations to inform DOL’s and the Department of Education's (ED) selection of one or more final measures. Based on these considerations, the authors also assess where further study of aspects of measuring the effectiveness in serving employers may be useful to consider by the research arms of DOL and ED.

Download this Summary (PDF)

Research Questions

• How do states, local areas and federal agencies define 'services to employers'? In what ways are the definitions similar and in what ways are they different?
• What measures are currently being used, or have been used in the past or in other contexts, to understand engagement with employers by states, local areas and Federal agencies? How are the data for these measures collected?
• What are options for evaluating the feasibility, validity and reliability of proposed measures?

Key Takeaways

• Performance data submitted by states in Program Year 2018 indicate that states were most commonly piloting the Retention with the Same Employer and Employer Penetration Rate measures: 42 states were piloting each of these measures.
• Interviewed state and local staff reported that the categories of employer services used for reporting on employer services during the pilot were adequate for performance reporting but had some limitations.
• For each measure, the study team identified threats to validity, reliability, and practicality as well as potential unintended consequences as required as part of the scope of work to assess the pilot measures.
• Despite the pros and cons for each measure, the study found few alternative measures that appear preferable.
• Though the authors did not find an overwhelming case for adopting either one measure or several measures, adopting more than one measure offers the advantage of capturing more aspects of performance and may make results more actionable for the different Title I, II, III, and IV programs.

Research Gaps

• Findings from our research highlight several options for future study: Further Analyze Selected Measure or Measures to Strengthen Them, Support Their Adoption, and Establish Satisfactory Levels of Performance. Examine Alternative Measures and Approaches That Could Supplement Adopted Measures and Improve Performance Measurement. Explore Alternative Data from the Census Bureau, Private Vendors, and the States as Potential Sources for Validating Results and Supporting State Data Collection Efforts. Develop Adjustment Models and Baseline Standards to Meet Federal Requirements and Improve Usefulness of Adopted Measures. (page 76)

Citation

Spaulding, S., Barnow, B., Briggs, A., Trutko, J., Trutko, A., Hecker, I. (2021). Urban Institute. Measuring the Effectiveness of Services to Employers: Options for Performance Measures under the Workforce Innovation and Opportunity Act (Research Report). Chief Evaluation Office, U.S. Department of Labor.

Download Report   View Study Profile

The Department of Labor’s (DOL) Chief Evaluation Office (CEO) sponsors independent evaluations and research, primarily conducted by external, third-party contractors in accordance with the Department of Labor Evaluation Policy and CEO’s research development process.