America’s Promise Job-Driven Grant Program Evaluation

About the Study

In 2017, the DOL’s Chief Evaluation Office (CEO), in collaboration with ETA, funded contractor Mathematica to conduct an implementation and impact evaluation of the America’s Promise grant program. The implementation study examines how the 23 grantee organizations implemented their programs between 2016 and 2020. Early findings on how the grantees’ sector training strategies were affected by the onset of the COVID-19 pandemic were released in 2021. Final findings from the evaluation’s impact study are anticipated later in 2023. This evaluation adds to the growing evidence base about how career pathways and related sector-based training programs and regional partnership strategies meet employers’ needs and prepare American workers for middle- and high-skilled jobs.

The America’s Promise implementation evaluation included the collection of qualitative data from a variety of sources — including virtual site visits, phone interviews with grant representatives and employer partners, focus groups with America’s Promise participants, a grantee survey, and a partner network survey — and analysis of grantees’ performance data from the Workforce Integrated Performance System (WIPS). Researchers used these data to identify the approaches taken by America’s Promise grantees to strengthen or establish regional partnerships and provide assistance to unemployed and underemployed individuals, as described in the final report. Additional reports include an evaluation design pre-specification plan, a data-use agreement toolkit, and three short issue briefs. The issue briefs describe in-depth findings from interviews conducted with a large number of employer representatives (31) as well as the survey administered across the 23 grantee sites. Finally, a short paper examines the outcomes of America’s Promise participants by gender after the onset of the COVID-19 pandemic.

This DOL-funded study was a result of the annual process to determine the department’s research priorities for the upcoming year. It contributes to the labor evidence-base to inform employment and training programs and policies and addresses Departmental strategic goals and priorities.

Download the summary

The Impact of Regional Sectoral Training Partnerships: Findings from America’s Promise

Most participants received at least one credential through the America’s Promise program (75 percent) and completed their training program (80 percent).
Compared to Wagner-Peyser participants with similar demographics and earnings history, participants in America’s Promise had a 6 percentage point higher employment in the fourth quarter after program enrollment and a 4 percentage point higher employment in the eighth quarter after enrollment, both statistically significant.
Relative to the same group of Wagner-Peyser participants, participants in America’s Promise had a $2,697 higher total earnings in the second year after program enrollment. This effect was statistically significant.

Employment Research Brief: The Impact of Regional Sectoral Training Partnerships: Findings from America’s Promise

Most participants received at least one credential through the America’s Promise program (75 percent) and completed their training program (80 percent).
Compared to Wagner-Peyser participants with similar demographics and earnings history, participants in America’s Promise had a 6 percentage point higher employment in the fourth quarter after program enrollment and a 4 percentage point higher employment in the eighth quarter after enrollment, both statistically significant.
Relative to the same group of Wagner-Peyser participants, participants in America’s Promise had a $2,697 higher total earnings in the second year after program enrollment. This effect was statistically significant.

Creating and Expanding Regional Workforce Partnerships for Skill H1-B Industries and Occupations: Implementation of America’s Promise Job-Driven Training Grants (final report)

America’s Promise grants led to many new employer partnerships. Although America’s Promise partnerships built on prior efforts, the 23 grantees reported that 74 percent of the total 593 employer partners were newly established for the grant.
Respondents from grantee organizations, partners, and employers highlighted the importance of prioritizing employer voices. Higher degrees of employer engagement appeared to ensure that services available through America’s Promise funded programs demonstrated employer needs and could help participants enter target industries.
Support programs were viewed as critical by program participants. Many program participants cited wraparound services as a differentiator for America’s Promise programs. Participants noted that case management and work readiness programs offered in tandem with training programs seemed to help ensure success for participants.
Strong partnerships were critical to providing services to participants. The emphasis of the America’s Promise program on strong partnerships was considered one the most important practices, according to grantees. These efforts reduced competition between entities with similar missions in the same region, such as community colleges, to better meet participant and employer needs.
Trainings offered by America’s Promise grantees aligned with employer needs. This alignment stemmed from employers assisting in program planning and helped program participants meet the hiring requirements of high-wage jobs. Participants’ needs were simultaneously addressed in some cases through earn-and-learn models such as apprenticeships which provided income and hands-on-experience.

Employer Perspectives on Regional Workforce Partnerships: Lessons from America’s Promise (issue brief)

Employers’ most reported reason for participating in America’s Promise partnerships was to have a voice in shaping workforce system strategies. Employers also stated that they were motivated to participate in America’s Promise partnerships to strengthen talent pipelines, fund training development and work-based learning, show commitment to their communities, and diversify their workforces.
Employers contributed to partnership activities by shaping partnerships activities, supporting work-based learning, and providing job placement. Although employer partners played multiple roles, they most often served on advisory boards, supported work-based learning opportunities through apprenticeships and on-the job-training, and employed individuals who completed training.
Employers identified areas of value and for improvement. Employers perceived regional partnerships as successful in supporting high quality training, improving employee retention, increasing industry growth and collaboration, and strengthening and diversifying the pipeline of skilled workers. Some employers also identified a need for more training funds, increased tailoring of programs, and improvement of childcare and transportation support for participants.
Most employers plan to continue playing a role in workforce development in their regions. The need to collaborate to advance workforce development was stressed by 30 of the 31 respondents. Four employers planned to continue training activities established under the America’s Promise grant; 26 others will seek additional funds to sustain training activities.

Employer Engagement Strategies in Regional Partnerships: Lessons from America’s Promise (issue brief)

Regional partnerships used multiple methods to successfully recruit at least 5 employer partners. Grantees typically established contact by phone, email, or via ongoing meetings of industry associations and chambers of commerce, which were followed by one-on-one conversations. Grantees recruited employers as partners for input on services and oversight or for work-based learning opportunities and job placement. Over half (58%) of employer partnerships were newly established as a part of the America’s Promise grant and grantees reported an average of 21 employer partners.
All regional partnerships sought employer partners willing to hire participants. Of the 18 partnerships visited virtually for the study, 12 had developed arrangements with employers that committed to hiring multiple participants. Of the same 18, 14 offered work-based learning opportunities, 7 offered apprenticeships and internships, 7 offered paid internships, 5 offered on-the-job training, and 2 offered pre-apprenticeships.
Interviews with program partners highlighted the importance of devoting sufficient time and resources to employer outreach and using incumbent worker training to engage employers. Other key lessons learned were the importance of engaging small businesses, timing outreach with the grant life cycle, reviewing regional employers when selecting an industry focus, engaging in small- and large-scale hiring partnerships, and establishing strong relationships with business services coordinators.

Experiences of America’s Promise Participants During the “Shecession”: Examining Gender Differences in Labor Market and Training Program Outcomes (short paper)

Both men and women participating in America’s Promise programs experienced a drop in employment and earning at the outset of the COVID-19 pandemic. Declines in employment and earnings did not significantly differ between men and women, with both groups experiencing a 9-percentage point decrease in employment rates.
Women participating in America’s Promise programs faced competing employment effects from the COVID-19 pandemic. Women were more likely to be trained in industries that were less impacted by the COVID-19 pandemic (e.g., health care). However, there was evidence that childcare responsibilities may have affected some female participants’ ability to work.
COVID-19 correlated with decreased training completion rates. Completion rates for America’s Promise training programs declined by 5 percentage points at the onset of the COVID-19 pandemic, but these declines were similar for both men and women.
The average completion time for training programs increased by almost a month after the onset of COVID-19. On average, the amount of time it took participants to complete a training program increased by 0.9 months after the onset of COVID-19. However, this varied by gender: men experienced an average completion time increase of 1.9 months while women experienced a decrease of 0.3 months. This may be due to differences in enrollment patterns across training industries.

Final Outcomes and Impact Report

Spitzer, A., English, B., Williams, B., Thal, D., Marks-Anglin, A., Weiss, C., Bellotti, J., Berk, J. (2023). Mathematica. The Impact of Regional Sectoral Training Partnerships: Findings from America’s Promise. Chief Evaluation Office, U.S. Department of Labor.

Employment Research Brief

Spitzer, A., English, B., Williams, B., Thal, D., Weiss, C., Forde, J., Bellotti, J., Berk, J. (2023). Mathematica. The Impact of the America’s Promise Sectoral Training Partnerships. Chief Evaluation Office, U.S. Department of Labor.

Final Implementation Study Report

English, B., Ochoa, L., Krantz, A., Rosenberg, L., Zelenack, S., Bart, E., Bellotti, J., Allmang, S., and Dunham K. (2022). Mathematica. Creating and Expanding Regional Workforce Partnerships for Skilled H-1B Industries and Occupations: Implementation of America’s Promise Job-Driven Training Grants. Chief Evaluation Office, U.S. Department of Labor.

Brief

English, B., Zelenack, S., and Bellotti, J. (2022). Mathematica. Employer Perspectives on Regional Workforce Partnerships: Lessons from America’s Promise. Chief Evaluation Office, U.S. Department of Labor.

Brief

Dunham, K., Folsom, L., and Bellotti, J. (2022). Mathematica. Employer Engagement Strategies in Regional Partnerships: Lessons from America’s Promise. Chief Evaluation Office, U.S. Department of Labor.

Brief

Spitzer, A., Berk, J., and Marks-Anglin, A. (2022). Mathematica. Experiences of America’s Promise Participants During the “Shecession”: Examining Gender Differences in Labor Market and Training Program Outcomes. Chief Evaluation Office, U.S. Department of Labor.

Evaluation Design Plan

Mathematica. (2022). DOL Evaluation Design Pre-Specification Plan: America’s Promise Job Driven Grant Program Evaluation. Chief Evaluation Office, U.S. Department of Labor.

Technical Appendix

Spitzer, A., Berk, J., and Marks-Anglin, A. (2022). Mathematica. Technical Appendix for Experiences of America’s Promise Participants During the “COVID-19 Recession”: Examining Gender Differences in Labor Market and Training Program Outcomes. Chief Evaluation Office, U.S. Department of Labor.

Toolkit

Ruggiero, R., Lewis, G., Bond, S., Deutsch, J., Geckler, C., Bellotti, J., Lieser, M., and Millar, A. (2022). Mathematica. Obtaining Administrative Data for DOL Research and Evaluations, Insights from Employment and Training Research. Chief Evaluation Office, U.S. Department of Labor.

One-Pager

Mathematica. (2022). America’s Promise Job-Driven Training Evaluation. Chief Evaluation Office, U.S. Department of Labor.

Brief

Bellotti, J., English, B., Harrington, A. (2021) Mathematica. Lessons from America’s Promise Partnerships: Sector Training Strategies During the COVID-19 Pandemic. Chief Evaluation Office, U.S. Department of Labor.

Project Duration: 87 Months

Contract End Date: September 2024

Contractor: Mathematica Policy Research and Social Policy Research Associates

For More Information: ChiefEvaluationOffice@dol.gov

The Department of Labor’s (DOL) Chief Evaluation Office (CEO) sponsors independent evaluations and research, primarily conducted by external, third-party contractors in accordance with the Department of Labor Evaluation Policy. CEO’s research development process includes extensive technical review at the design, data collection and analysis stage, including: external contractor review and OMB review and approval of data collection methods and instruments per the Paperwork Reduction Act (PRA), Institutional Review Board (IRB) review to ensure studies adhere to the highest ethical standards, review by academic peers (e.g., Technical Working Groups), and inputs from relevant DOL agency and program officials and CEO technical staff. Final reports undergo an additional independent expert technical review and a review for Section 508 compliance prior to publication. The resulting reports represent findings from this independent research and do not represent DOL positions or policies.