Department of Labor Evaluation Design Pre-Specification Plans (Issue Brief)

« Back to Publication Search Results
Release Date

Department of Labor Evaluation Design Pre-Specification Plans (Issue Brief)

Download Report
Issue
2023-06

Publication Info

The Department of Labor’s (DOL’s) Chief Evaluation Office (CEO) is committed to upholding the department’s Evaluation Policy principles of rigor, relevance, transparency, and ethics in independent evaluations. For all rigorous experimental studies and studies using methods described as quasi-experimental, CEO will publish Evaluation Design Pre-Specification Plans during the planning stages to promote transparency, and replicability. It is important to note that changes may occur during the research stage after the publication of Design Plans, and final evaluation products will clearly note where and why research altered from published plans.

This document provides a template that evaluators must use to meet the pre-specification practices articulated in OMB Memo M-20-12 Phase 4 Implementation of the Foundations for Evidence-Based Policymaking Act of 2018: Program Evaluation Standards and Practices. OMB Memo M-20-12 calls for making an "evaluation’s design and methods available before the evaluation is conducted and in sufficient detail to achieve rigor, transparency, and credibility by reducing risks associated with the adoption of inappropriate methods or selective reporting of findings, and instead promoting accountability for reporting methods and findings." The information reported must also provide sufficient information that final reporting could be assessed per the DOL Clearinghouse for Labor Evaluation and Research evidence guidelines. For two decades, DOL has invested in reentry services by committing substantial funding toward programs serving justice-involved young adults and adults, under the Reentry Projects (RP) grants. Between 2016 and 2019, DOL awarded almost $300 million through these grants to improve participants’ labor market and criminal justice outcomes. Grantees include both intermediary organizations that serve large numbers of participants across multiple sub-grantees and states, and smaller, community-based organizations. The services offered vary depending on the grant stream and target group, but all offer career preparedness, employment-focused services, and case management. In addition, all RP grantees were required to use at least one of the following employment strategies: registered apprenticeship, work-based learning, and career pathways.

CEO contracted with Mathematica and Social Policy Research Associates to build evidence about effective strategies to serve people with prior justice involvement and facilitate their successful reentry into the community. This evaluation aims to determine the impacts of the program on labor market and criminal justice outcomes (impact study), understand how the grant programs were implemented across a broad range of intermediaries and CBOs (implementation study), and measure the outcomes of a broader set of RP participants than those included in the impact study (outcomes study). This document describes the quasi-experimental design (QED) of the impact study.