About the Report

Download Report

In June 2013, the U.S. Department of Labor (DOL) contracted the implementation of the “Feasibility Study and Evaluation of the Non-Traditional Occupation (NTO) Demonstration.” NTOs for women generally offer higher wages and more opportunities for advancement compared to traditionally female dominated occupations. The overall purpose of the NTO study was to: (1) identify barriers to NTOs and promising strategies for addressing those barriers, (2) design and implement a demonstration of a promising strategy, and (3) evaluate the effectiveness of the strategy implemented under the demonstration. The evaluation team engaged two American Apprenticeship Initiative (AAI) grantees providing entry-level training in NTOs— Construction and Advanced Manufacturing in South Seattle and Information Technology in New Mexico— to implement the demonstration

The intervention involved multi-mode outreach designed to reduce misperceptions, increase awareness and encourage women to enter traditionally male occupations and related training programs. To evaluate the intervention, a study sample comprised of female jobseekers engaged with the local public workforce system was utilized. The research team initiated a randomized-control trial (RCT), in which a treatment group received gender-themed recruitment content designed to increase awareness of NTOs among women and to address misperceptions women may have about NTOs. A control group received non themed, generic recruitment content. Content was delivered to individuals assigned to each group through multiple emails and a postcard, all of which directed the jobseeker to a website containing additional recruitment content. Each contact encouraged the jobseeker to take specific steps towards enrolling in an NTO training program. The evaluation design measured and compared the effectiveness of the two sets of recruitment content used to encourage women to take short-term actions towards applying to an NTO training program. The actions or behaviors female job seekers were nudged to take included: (1) opening the email, (2) clicking on a link in an email directing the jobseeker to a recruitment website, (3) completing an interest form, (4) gathering more information on the training program, and (5) applying for the training program.

The impact study found statistically significant positive effects in South Seattle on outcomes 2 (clicked on a link in an email) and 3 (completed an interest form). In New Mexico, the evaluation team observed a change in magnitude and direction of the impact estimates from negative and statistically significant for outcome 1 (opened the email) to nearly zero for outcome 3. Completion rates for the two exploratory outcomes—outcomes 4 (looked for more information) and 5 (enrolled in the program)—were low overall in the two demonstration sites as well as for both treatment and control groups within sites.

Analysis of outcome completion rates for both treatment and control group over time suggested that the sharp increases in outcome completion following each reminder constitute promising evidence supporting the effectiveness of a multi-pronged recruitment approach.

The report concludes with a discussion of study findings, implications, recommendations, and next steps for future research.

Research Questions

• Is customized/women‐themed recruitment content more effective in encouraging women to take short‐term steps toward applying to an NTO training program compared to non‐customized/non‐themed, generic recruitment content?
• Do the differences in responses to these messages vary across different subgroups of women (age, race, ethnicity, education, etc.)?

Key Takeaways

• Implemented in two sites reflecting a range of socio-economic and programmatic differences, the intervention’s impact differed by site. Individually, and in some cases together, the site findings point to important implications of evidence-based methods of conducting outreach that can encourage women to take early steps towards NTO participation.
• Overall, gender-themed content can be effective, and repeated exposure to the content through multiple modes and multiple “touches” may be key to increasing participation.

Citation

Nanda, N., Corea, C., Patterson, L., Poe Yamagata, E., Mian, P., Zhang, C. (2018). IMPAQ International. Feasibility Study and Evaluation of Non-Traditional Occupation Demonstrations: Final Evaluation Report. Chief Evaluation Office, Employment and Training Administration, U.S. Department of Labor.

Download Report

This study was sponsored by the Employment and Training Administration, Office of Policy Development and Research, Division of Research and Evaluation, and was produced outside of CEO’s standard research development process.