Reemployment Services and Eligibility Assessments (RESEA) Evidence Building and Implementation Study

• Evidence Building at the U.S. Department of Labor: Lessons From Delivering Reemployment Services and Eligibility Assessment (RESEA) Evaluation Technical Assistance (Issue Brief, April 2024)
  • Summary
• Tracking Changes in Program Implementation: Findings from Multiple Rounds of the Reemployment Services and Eligibility Assessment (RESEA) Implementation Survey (Issue Brief, April 2024)
  • Tracking Changes in Program Implementation: Findings from Multiple Rounds of the Reemployment Services and Eligibility Assessment (RESEA) Implementation Survey Appendix: Detailed Wave 4 Survey Results (Appendix, April 2024)
  • Summary
• Cost-Benefit Analysis for RESEA Programs: A Handbook (Handbook, April 2024)
• An Introduction to RESEA Component Evaluations (Webinar, November 2022)
• RESEA Peer Learning (Webinar, August 2022)
• RESEA Program Strategies: State and Local Implementation (Final Report, June 2022)
  • Summary
• Serving Unemployment Assistance Claimants During a Public Health Crisis: The RESEA Program and the COVID-19 Pandemic (Issue Brief, June 2022)
  • Summary
• Reemployment Services Evidence: A Collection of Briefs on RESEA Program Components (Compendium, June 2022)
  • Summary
  • Brief 1: Selecting Claimants and Meeting Attendance
  • Brief 2: Basic Career Services
  • Brief 3: Individualized Services
• Options for Building Evidence on RESEA Programs (Final Report, June 2022)
  • Summary
• RESEA Program Strategies: State and Local Implementation (Webinar, April 2022)
• Live Q & A: REA Impact Study Findings and Implications for RESEA Programs and Evaluations (Webinar, October 2020)
• Forming Consortia to Evaluate RESEA Programs (Webinar, January 2020)
• RESEA: Overview of Evaluation and Evidence Requirements (Webinar, November 2019)
• Evaluating RESEA: How Does it Help My State and Where Do We Start? (Webinar, May 2019)
• Evaluation Information and Resources for RESEA Programs (Webinar, January 2019)
• FY 2019 RESEA Grants: New Program Requirements and Application Instructions (Webinar, January 2019)
• Evaluation Technical Assistance Resources
  • RESEA Evaluation Toolkit
  • RESEA Evaluation Resource List
• Related Links and Collaborations
  • RESEA Evaluation and Evidence Resources on WorkforceGPS
  • CLEAR’s RESEA Topic Area Tab

• What options for developing and applying tiered evidence standards to RESEA interventions should be considered? What approaches for identifying evidence-based RESEA service delivery strategies and practices should be considered?
• What is the current state of the evidence of RESEA and its components? How is RESEA being implemented? What are the types and packages of services provided under current RESEA programs? What common components of RESEA programs are effective or promising?
• What options should DOL and states consider for building the evidence base and filling gaps in the literature?

Evidence Building at the U.S. Department of Labor: Lessons from Delivering Reemployment Services and Eligibility Assessment (RESEA) Evaluation Technical Assistance

• The RESEA legislation included several provisions that laid the groundwork for evidence use and evidence building to play a central role in the program. A unique and important aspect of RESEA is that rigorous evaluation was explicitly recognized as a core element of the program as part of the authorizing 2018 amendment to the Social Security Act. Moreover, program funding is tied to the use of well-conducted, credible impact studies that inform the design of states’ RESEA programs.
• DOL has experience, capacity, and an infrastructure that it was able to draw from and build upon to support RESEA evaluations and evidence building. DOL has a strong evidence-building culture and has since 2010 been undertaking activities—such as having evaluation officers, developing learning agendas, and establishing an evaluation policy—now required of most federal agencies by the Evidence-Based Policymaking Act. Additionally, DOL has a clearinghouse that conducts evidence reviews and assigns causal evidence ratings in the way required by the new law.
• Federal-level leadership and cross-agency coordination, across the program and evaluation offices in providing evaluation technical assistance, was considered critical to the progress made by states in developing and launching evaluations to build evidence. The Chief Evaluation Office and the Employment and Training Administration’s Offices of Unemployment Insurance and Policy Development and Research collaborated closely to support states in meeting requirements of the law, including beginning new evaluation efforts.

Tracking Changes in Program Implementation: Findings from Multiple Rounds of the Reemployment Services and Eligibility Assessment (RESEA) Implementation Survey

• States consistently reported that they targeted claimants identified as most likely to exhaust Unemployment Insurance (UI) benefits when selecting UI claimants for the RESEA program. While states also used other criteria when selecting UI claimants for the RESEA program, they prioritized those who are most likely to exhaust UI benefits before and after the COVID-19 pandemic.
• The timing of when the initial RESEA meeting takes place, relative to the notification of selection, remained relatively consistent between Waves 1and 4. Across the four waves, the initial RESEA meeting most often occurred within two weeks after notification of RESEA selection regardless of when in the UI claim RESEA participants were selected for the program. In Wave 4, a larger percentage of states reported selecting claimants early into their claim period.
• Overall, states appeared to be providing more flexibility to claimants in scheduling RESEA meetings and in the location of those meetings than they did prior to the pandemic. Since the onset of the COVID-19 pandemic, the use of remote service delivery options, including phone calls and videoconferences, increased. Despite increased flexibility with mode of service delivery, the content and services provided during the initial RESEA meetings remained similar in all waves (fielded between 2020-2023).

RESEA Program Strategies: State and Local Implementation (Final Report)

• RESEA programs vary substantially across the country. States use various strategies for promoting RESEA meeting attendance and reducing the rate of participants that fail to attend. The subject matter of RESEA meetings also varies by state and meetings may involve staff from other public workforce programs instead of or in addition to RESEA dedicated staff.
• Staff noted few changes from 2018 to the onset of COVID but indicated interest in more changes. These changes included implementing strategies to enhance reemployment services and changes to the criteria used to select UI claimants for RESEA to include claimants at lower risk of exhausting their UI benefits.
• Eligibility issues identified when meeting with claimants are often not adjudicated. Staff interviews and survey responses indicated that enforcing UI eligibility requirements often made it more difficult to establish necessary trust with claimants. For this reason, staff often prioritized educating claimants of requirements and job search strategies rather than using meetings as enforcement activity.
• RESEA service delivery changed dramatically post-COVID. UI applications surged as a result of COVID, with agencies reassigning RESEA staff to process new applications. Delivery of RESEA services shifted from in-person to remote. The number of claimants served decreased precipitously early in the pandemic, recovering (though not completely) in late 2020.
• Most states are making progress toward launching evaluations and interest exists for testing new strategies. Most state administrators expressed an understanding of and support for evidence-building and performance requirements. Three-fifths of states were confident in their capacity to design, administer, and undertake evaluations of their RESEA programs.

Serving Unemployment Insurance Claimants During a Public Health Crisis: The RESEA Program and the COVID-19 Pandemic (Issue Brief)

• Three-quarters of state RESEA programs temporarily suspended operations beginning in March 2020. This was accompanied by a precipitous decline in the number of RESEA participants at the start of the pandemic. A year later, all but one state had restarted their RESEA programs.
• Operations recovered late in 2020, but participation remained well below pre-COVID levels. The number of scheduled and completed meetings for the RESEA program did not reflect earlier figures prior to the pandemic.
• Remote service delivery was utilized to deliver RESEA services. The remote service model was widely adopted by the spring of 2021, with four-fifths of states indicating that initial RESEA meetings were typically conducted remotely by phone. Remote and virtual services are likely to remain as RESEA features after the pandemic.
• Claimant selection remained consistent during the pandemic, including the timing of claimant selection and initial meetings. A few states reported changes to work search review.

Reemployment Services Evidence: A Collection of Briefs on RESEA Program Components (Compendium)

• Available evidence suggests that the statistical likelihood that a claimant will exhaust UI benefits is not the best predictor of program impact. This is a common method of participant selection, but other claimant characteristics, such as their weekly benefit amount at the time of enrollment, may more accurately predict impacts of program participation.
• Additional evidence is needed to robustly demonstrate the effectiveness of several RESEA program components. For example, most RESEA programs incorporate both basic and individualized career services, and while there is evidence that individualized and intensive services improve participant outcomes, this evidence may not align to the conditions under which RESEA programs are implemented. In addition, mandatory participation requirements appear relevant to program impacts, but few studies have isolated the effect of reemployment services from the mandate to participate.
• Researchers identified priority evaluation options for the short and long term, as well as how those options align in an evidence-building strategy. The evaluation options are related to whole program evaluations, participant subgroups, or program components. For each option, researchers showed whether it is likely to be led by states or by DOL, how soon the activity might start and end, and a rough estimate of cost.
• Evaluation timelines imply that new RESEA-specific impact results will not be available until after evidence-based requirements are in effect. Some requirements are currently in effect, while others take effect in FY2023. In contrast, the earliest effectiveness results may be approved in time for state FY2025 or 2026 RESEA Plans, and based on the chosen design, many states’ evaluations may yield results in time for state FY2027 or 2028 RESEA Plans.
• Longer-term evaluation studies should build on short-term designs to yield results by the early 2030s. These studies require more preparation, including developing sufficiently large sample sizes and piloting the program or program component to be studied.

Options for Building Evidence on RESEA Programs (Final Report)

• The evidence-building options report serves as a resource for decision makers to understand and weigh options for developing evidence of various types on various timelines. Specifically, for a wide range of options, the document considers: What should be evaluated? How should that be evaluated? Who (states, consortia of states, or DOL) should initiate the particular evaluation? When should that evaluation occur? And how can the state workforce agency’s capacity to conduct evaluations be strengthened?
• Research may be funded and directed by individual states, a consortium of states, or the Department of Labor (DOL). States’ input and management is crucial: they administer the programs and most of the evaluation funds. The DOL can also leverage its national perspective and awareness of state RESEA plans to catalyze and support inter-state coordination when required, such as to execute design options that require larger samples than most states can develop independently.
• Evidence building challenges exist, but DOL is well-positioned to help states address them: Evidence-building challenges across states include: the need for very large sample sizes to conduct adequately powered impact studies; program variation across states may make partnering on consortia evaluations more difficult; states have an intrinsic incentive to not conduct their own evaluations and instead ‘free ride’ on others’ evaluations; generalizability challenges make it difficult for states to know which evidence to use when informing their own programs; states may be (knowingly or unknowingly) conducting studies that are not capable of providing high quality evidence that demonstrates program effectiveness; the RESEA legislation incentivizes serving more people by providing outcome payments based on number of participants served, which disincentivizes random assignment designs that require a control group; and, the emerging timelines states are on to complete evaluations and produce new causal evidence is not timely to meet FY23 requirements. These challenges imply DOL has an important role in setting appropriate policies; providing evaluation technical assistance to states; facilitating inter-state coordination; and developing new evidence that takes advantage of results from across states in support of the nationwide program.

Issue Brief
Copson, E., Irwin, M., Martinson, K., Engle, H., Wheat, V., Epstein, Z., Cuellar, M. (2023). Abt Associates. Evidence Building at the U.S. Department of Labor: Lessons From Delivering Reemployment Services and Eligibility Assessment (RESEA) Evaluation Technical Assistance. Chief Evaluation Office, U.S. Department of Labor.

Issue Brief
Epstein, Z., Hawthorne, A., Copson, E. (2023). Abt Associates. Tracking Changes in Program Implementation: Findings from Multiple Rounds of the Reemployment Services and Eligibility Assessment (RESEA) Implementation Survey. Chief Evaluation Office, U.S. Department of Labor.

Appendix
Epstein, Z., Hawthorne, A., Copson, E. (2023). Abt Associates. Tracking Changes in Program Implementation: Findings from Multiple Rounds of the Reemployment Services and Eligibility Assessment (RESEA) Implementation Survey Appendix: Detailed Wave 4 Survey Results. Chief Evaluation Office, U.S. Department of Labor.

Toolkit
Dastrup, S., Harvill, E., Klerman, J.A. (2023). Abt Associates. Cost-Benefit Analysis for RESEA Programs: A Handbook. Chief Evaluation Office, U.S. Department of Labor.

Reemployment Services and Eligibility Assessments (RESEA) Evidence Building and Implementation Study

RESEA Program Strategies Report
Trutko, J., Trutko, A., Clarkwest, A., Souvanna, P., Klerman, J., Briggs, A., Spaulding, S., Hecker, I., Islam, A., Katz, B., Scott, M., Nightingale, D. (2022). Abt Associates. RESEA Program Strategies: State and Local Implementation. Chief Evaluation Office, U.S. Department of Labor.

Issue Brief
Abt Associates. (2022). Serving Unemployment Insurance Claimants During a Public Health Crisis: The Reemployment Services and Eligibility Assessment (RESEA) Program and the COVID-19 Pandemic. Chief Evaluation Office, U.S. Department of Labor.

Compendium
Epstein, Z., Klerman, J.A., Clarkwest, A., Nightingale, D. (2022). Abt Associates. Reemployment Services Evidence: A Collection of Briefs on RESEA Program Components. Chief Evaluation Office, U.S. Department of Labor.

Selecting Claimants Brief
Epstein, Z., Klerman, J.A., Clarkwest, A., Nightingale, D. (2022). Abt Associates. Selecting Claimants and Meeting Attendance. Chief Evaluation Office, U.S. Department of Labor.

Basic Career Services Brief
Epstein, Z., Klerman, J.A., Clarkwest, A., Nightingale, D. (2022). Abt Associates. Basic Career Services. Chief Evaluation Office, U.S. Department of Labor.

Individualized Services Brief
Epstein, Z., Klerman, J.A., Clarkwest, A., Nightingale, D. (2022). Abt Associates. Individualized Services. Chief Evaluation Office, U.S. Department of Labor.

Options for Building Evidence Report
Klerman, J.A., Nightingale, D., Clarkwest, A., Epstein, Z. (2022). Options for Building Evidence on RESEA Programs. Chief Evaluation Office, U.S. Department of Labor.