CEO Library

Ongoing research and evaluation that produce actionable, widely disseminated evidence are central to the continuous improvement of U.S. Department of Labor (DOL) programs. In addition, the Workforce Innovation and Opportunity Act of 2014 (WIOA) requires DOL to conduct periodic, independent evaluations to inform the effective operation of WIOA programs and services. To support the development of DOL’s research portfolio, this scan examines existing evidence on key topics related to WIOA programs and services.

The summary details the research activities and highlights key findings from all components of the five-year Performance Partnership Pilots for Disconnected Youth (P3) evaluation. First, researchers describe the implementation study, and provide an overview of the study’s findings. Then, they describe the evaluation technical assistance activities provided to grantees and their local evaluators and present findings from the synthesis of Cohort 1 pilots’ local evaluation reports. Lastly, they offer brief considerations of how the lessons learned from P3 can inform future efforts.

In the brief, the Partnership Pilots for Disconnected Youth (P3) study team has placed pilots’ efforts to sustain systems change along a continuum. At one end of this continuum, two pilots approached P3 as a platform to facilitate systems change in their communities. Next along the continuum is a pilot that had taken initial steps toward systems change by the end of its P3 grant. Next, two pilots reported that through P3 they had strengthened partnerships and broken down silos but that the systems for serving disconnected youth did not experience much change as a result of P3.

The brief provides a case study of the Sacramento Partnership Pilots for Disconnected Youth (P3) pilot. As such, it seeks to understand the context for the pilot and describe the pilot’s approach to supporting homeless youth. First, drawing on current literature, the brief provides an overview of the state of youth homelessness and housing insecurity in Sacramento, along with a review of approaches that may have promise in supporting this population.

The Consolidated Appropriation Act of 2014 authorized the Performance Partnership Pilots for Disconnected Youth (P3), which provided awarded pilots the flexibility to use funding from across multiple Federal discretionary programs to support efforts to improve the systems serving youth and youth’s outcomes. The report assesses the 14 awarded pilots’ implementation of the Federal vision for P3. Findings showed that pilots took a variety of approaches to try to improve youth outcomes, which commonly included new or enhanced services.

COVID-19 has caused hiring freezes and business and institutional closures, which affected disconnected youth’s ability to continue working with service providers to meet employment and education goals and basic needs. In response, and in order to continue supporting youth, providers have adapted their services. To assess these adaptations, Mathematica and its subcontractor, Social Policy Research Associates, conducted a supplemental study as part of the National Evaluation of the Performance Partnership Pilots for Disconnected Youth (P3).

Many youth in America are not on track for labor market success. One factor that increases the risk of poor labor market outcomes among these youth is dropping out of school (Rumberger 2020). Youth who drop out of school are at greater risk for job instability and for lower long-term earnings (Hair et al. 2009). They are also more likely to struggle with mental health and substance abuse issues (Maynard et al. 2015). These challenges are compounded for youth who have early involvement with the juvenile or criminal justice systems.

The brief examines one aspect of the Job ChalleNGe grants – the goal of serving more court-involved youth. It draws on findings from multiple data sources including two rounds of site visits, a background information form, a follow-up survey with Job ChalleNGe participants, program records, and postsecondary and criminal justice administrative records.

The report provides details on the data, samples, methods, and analyses for the Youth CareerConnect (YCC) impact study. Rigorously evaluating the effects of the YCC program on student outcomes required that multiple technical pieces be put in place, from selecting districts to participate in the evaluation to collecting and processing high-quality data and measuring impacts to conducting rigorous analysis to estimate impacts. This report provides details of these processes.

In 2014, the U.S. Department of Labor (DOL) awarded $107 million in four-year grants to 24 applicants for the Youth CareerConnect (YCC) program. At its core, the YCC program was designed to strengthen America’s talent pipeline by putting high school students on a pathway to complete postsecondary education, obtain industry-recognized credentials, and secure a job in middle- to high-skilled fields that often rely on foreign workers.

The report provides the findings from the impact study of Youth CareerConnect (YCC). The YCC impact study assessed short-term student outcomes with two rigorous components—a quasi-experimental design (QED) study in 16 school districts and a randomized controlled trial (RCT) in four school districts that were also in the QED.

The report examines information from a literature and database review that identified knowledge gaps, discussions with Wage and Hour Division (WHD) about compliance strategies, and discussions with a panel of experts about compliance strategies. The researchers then developed a framework for WHD and other agencies to consider when designing processes for monitoring and evaluating strategies and outcomes.

The report provides intervention and evaluation designs for two behavioral intervention (BI) trials that, if implemented, would test whether webinar registrations increase when behavioral strategies are applied to emails targeting a given industry. To design the study, researchers followed a six-step process developed by the U.S. Department of Labor (DOL) for designing and conducting BI studies that has resulted in trials that produce meaningful evidence.

The report supplements the Wage and Hour Division’s (WHD) Compliance Strategies Evaluation by exploring whether and how data that are housed outside of WHD could be integrated with WHD’s administrative data. The researchers explored how external data can be integrated with WHD’s case management system, the Wage and Hour Investigative Support and Reporting Database (WHISARD), and the limitations of doing so.

The U.S. Department of Labor (DOL)’s Chief Evaluation Office (CEO), in partnership with the Wage and Hour Division (WHD) contracted with Mathematica Policy Research to synthesize existing literature and data related to WHD compliance strategies.

The report is one in a series of implementation study papers of the Performance Partnership Pilots for Disconnected Youth (P3) National Evaluation, which was contracted by the U.S. Department of Labor’s (DOL) Chief Evaluation Office (CEO) on behalf of the Federal partners supporting P3. It assesses P3 four years after its initial authorization, and is based on data collected from interviews with Federal agency staff and two rounds of site visits to the nine original (Cohort 1) pilots, conducted between 2016 and 2018.

The report systematically documents and analyzes the use of technology-based learning (TBL) across the entire network of Job Corps centers, reporting on the TBL implementation process, the challenges TBL faced during implementation, and the best practices for TBL that instructors, managers, and directors identified.

The report—one component of the Performance Partnership Pilots for Disconnected Youth (P3) National Evaluation—synthesizes findings from the local evaluations of the Cohort 1 pilots. This report assesses the extent to which the local evaluations established a causal impact between the studied intervention and participant outcomes and, for interventions that had such evidence, whether the evidence indicated the intervention had improved outcomes for youth.

The brief provides reflections from four evaluation technical assistance (TA) liaisons based on their experiences in working with grantees awarded as part of a Federal interagency initiative, Partnership Pilots for Disconnected Youth (P3). The brief, (1) describes the P3 program’s TA supports, (2) reviews the roles and responsibilities of different types of P3 partners, and (3) describes the hurdles faced when working with partners and strategies to mitigate those hurdles.

The report from the Performance Partnership Pilots for Disconnected Youth (P3) evaluation’s implementation study reflects on the early experiences of the nine Cohort 1 pilots. The data primarily come from interviews with pilot stakeholders conducted in spring and summer 2017. Across the nine pilots, the evaluation team interviewed 169 stakeholders, including P3 administrators, staff, and partners. The report begins by describing P3 as envisioned by the Federal government, describes the nine pilots to provide context for the emerging findings, and then presents the early findings.

Administered by the U.S. Department of Labor (DOL), the Job Corps program trains approximately 60,000 participants on workforce skills in a residential environment each year. The Workforce Innovation and Opportunity Act requires DOL to present the cost per enrollee and cost per graduate of each Job Corps center every year. The report presents the methodology used to estimate cost per enrollee and cost per graduate for program year 2017. The average Job Corps center cost is $34,301 per enrollee and $57,312 per graduate.

In 2020, the Chief Evaluation Office (CEO) partnered with the Employment and Training Administration (ETA) and funded contractor Mathematica Policy Research to conduct the Workforce Innovation and Opportunity Act (WIOA) Research Portfolio Project.

In July 2012, Fort Scott Community College in Fort Scott, Kansas, received a grant totaling $1.5 million to develop and implement the water technologies training program for residents of the Kansas Juvenile Correctional Complex (KJCC) in Topeka. The goal of the training program was to increase incarcerated youth’s employment and earnings potential and reduce their recidivism rates after release from the correctional facility. Obtaining good job placements in high-demand occupations could help youth develop economic stability and, ultimately, self sufficiency.

The report, the second of the Youth CareerConnect (YCC) project’s implementation study, examines the evolution of YCC program implementation, and focuses on the third and fourth years of the grant, when grant funding was scheduled to end. It also examines grantee approaches to sustainability of YCC activities and services as they approached the end of grant funding.

In 2014, law enforcement agencies in the United States made nearly 1 million juvenile arrests. Roughly half of the cases formally processed resulted in youth being adjudicated delinquent. Youth with convictions face lasting collateral consequences such as decreased access to education, employment opportunities, and certain social welfare benefits, like Supplemental Nutrition Assistance Program (SNAP) or Temporary Assistance for Needy Families (TANF), aside from more direct consequences like fines, fees, or imprisonment. Recognizing this, the U.S.