DIRECTIVE	:	UNEMPLOYMENT INSURANCE PROGRAM LETTER NO. 13-95
TO	:	ALL STATE EMPLOYMENT SECURITY AGENCIES
FROM	:	MARY ANN WYRSCH Director Unemployment Insurance Service
SUBJECT	:	Field Test Results, Plans for Implementation of Unemployment Insurance (UI) Performance Measurement Review (PMR) Project, and Status of Oversight Efforts.

1. Purpose. To convey the results of a six-State field test of performance measures under the PMR Project and to provide States with the opportunity to comment on the interim final report.

2. References. Federal Register Notice (FRN) No. 54 FR 2238; FRN No. 57 FR 126; Unemployment Insurance Program Letter (UIPL) No. 10-89; UIPL No. 13-91; and UIPL No. 30-92.

3. Background. The Department of Labor (DOL) initiated the PMR Project in 1988 to examine, evaluate and improve performance measurement in the Unemployment Insurance Service oversight of State Employment Security Agencies (SESAs). The UI Quality Appraisal (QA) program had not been subject to in-depth review since its inception in the mid 1970s, so it did not take into account the impact of changing technologies in the delivery of UI services to the public. At the same time the Department recognized a need to integrate the components of its Federal oversight system.

   This UIPL provides current information regarding PMR design, field test results, and implementation plans.

4. PMR Design and Field Test. PMR was designed in three phases. Phase I consisted of the analysis of existing oversight measures to determine their legislative basis and to identify gaps or overlaps in measurement areas, the development of new or revised measures and the design for a field test of selected measures.

   With the involvement of State and Federal UI representatives, the project defined eleven timeliness measures and five quality measures covering benefit payments, adjudications, lower authority appeals, and the Combined Wage Claim (CWC) program. Some of the measures added program categories and reporting intervals to existing measures derived from the Unemployment Insurance Required Reports system; some of the measures looked at UI service areas which had not formerly been measured.

   In Phase II, six States successfully conducted a 15-month field test of PMR measures. The test ran from July 1993 through September 1994. UIS provided the Field Test States -- California, Illinois, Kansas, Missouri, New Hampshire, and Wisconsin -- with the PMR data entry applications on their existing UI system platform. UIS National Office designed a data base system to automate, as much as possible, the entry, storage, and analysis of the PMR data. The States developed programs to extract PMR report data from their mainframe-based unemployment compensation records. States also developed extract programs to select random samples from a universe of eligible cases.

   The first 3 months of the test were a pretest of system capabilities and State programming. Corrections were made during that time with the help of Mathematica Policy Research, Inc., (MPR) of Princeton, New Jersey. MPR contracted with the State of New Hampshire to assess State ability to implement the PMR design, to validate the data collected and to provide analysis and recommendations for establishing levels of performance.

   The "Interim Evaluation Report" dated December 5, 1994, contains an analysis of the field test data by MPR and is included as an attachment to this UIPL. The report shows that the new performance measures and the software developed could yield meaningful and statistically valid information in a cost-effective manner, except for the CWC quality measures. States are invited to review and comment on the attached "Interim Final Report," as comments will benefit the development of final implementation plans.

   In Phase III, PMR will be implemented nationwide.

5. PMR Implementation. After resolving comments from stakeholders responding to this UIPL, UIS will provide States by May 1995, detailed implementation directions including data systems specifications and an implementation schedule.

6. Status Report on UI Oversight. DOL's oversight programs have evolved over time. Each component represents a building block in the Federal UI oversight structure, beginning with Quality Appraisal (QA) and Workload Validation (WLV), and growing to include Cash Management, Benefits Quality Control (BQC) and Revenue Quality Control (RQC) programs. The Department recognized a need to integrate parts of the oversight system so that they would form a more coherent whole, and has convened a workgroup to examine the issue.

   1. Performance Enhancement Work Group (PEWG). The PEWG is a joint Federal/State work group composed of representatives from Federal National and Regional Offices, and State Agencies formed in conjunction with the Interstate Conference of Employment Security Agencies. The PEWG has been working for 13 months to revise systems for measuring and fostering good performance in the UI system. The group has developed a set of partnership principles which recognize mutual responsibility for the UI system. Shared ownership is made real by collaboratively involving State Agencies and other stakeholders in developing policy and procedures.

      A basic tenet of the PEWG is that all measures and standards developed for the UI system should be implemented in such a way as to foster continuous improvement, and that the comprehensive oversight system should recognize the respective interests of each partner. The PEWG has reviewed the measures the PMR project field tested and the measures that RQC is currently implementing. In coming months the PEWG will address ways to assign numeric standards for performance to replace the existing system of Desired Levels of Achievement and Secretary's Standards.

      The PEWG has specifically addressed three areas of PMR:

      Detection Date. The PEWG has suggested using the date the issue is detected as the starting point for measuring the promptness of adjudications. The PMR Field Test measured adjudication promptness from the week ending date of the first week affected by the adjudication to the date that the determination was issued.

      Redetermination Promptness. The PEWG has suggested that UIS not implement a measure of redetermination promptness. The attached "Interim Final Report" contains a discussion of the field test results of this measure.

      Combined Wage Claims Quality Measures. The PEWG has suggested that UIS not implement measures of Combined Wage Claims Quality. The attached "Interim Final Report" contains a discussion of the field test results of these measures.

      A decision on these issues, as with all the other issues surrounding implementation of the PMR measures, will be influenced by comments from interested stakeholders in the system who respond to this UIPL.

   2. Major Changes from Quality Appraisal (QA). QA currently provides an annual look at State performance. PMR results are available with more frequency, that is, monthly and quarterly rather than annually. PMR differs from QA in deriving data for performance measures from universe data wherever this data is available on States' automated systems. PMR measures also contain more reporting intervals and program categories. PMR universes are generally more comprehensive than those used for QA, including all first payments whether total or partial, for example.

   3. Data Validation. Data collected by DOL are used to support a number of important indicators that assess the nation's economic strength, the performance of States in the administration of the UI program, and to determine funding levels for State UI program administration. To ensure that the data reported by States adhere to DOL definitions and are counted properly, DOL is working toward a unified approach to data validation in benefits and tax program areas. Data validity is critical to sound UI program administration at the State and Federal levels.

      Workload Validation dates back to 1976. Although it has been modified at times to adjust to changing economic and legislative requirements, Workload Validation has not been reviewed in its entirety since its inception. Therefore, in 1992, UIS convened an internal work group that recommended two coordinated validation efforts: one effort to validate benefits data used for PMR, administrative financing, and certain economic data, and the other to review tax data used by the RQC program. Through the joint efforts of PMR and RQC initiatives, DOL is pursuing an integrated approach to data validation.

   4. Benefits Quality Control (BQC). BQC is currently being evaluated by a work group consisting of Federal National Office, Regional Office, and State UI representatives. The purpose of this evaluation work group is to determine the effectiveness and/or shortcomings of the entire BQC program to inform PEWG deliberations on the redesign of BQC. To that end, the work group has canvassed UI Federal and State personnel through the use of survey forms distributed at various staffing levels in these organizations. These results, along with available historical data, and other existing data, will be relied on heavily in producing BQC program findings and conclusions, and subsequently providing a report to the PEWG. This final report will reflect on all aspects of the program, both positives and negatives, and will include recommendations. The report is scheduled for delivery by the end of January 1995.

   5. Revenue Quality Control. The RQC program assesses State tax functions using systems reviews, acceptance samples and method surveys, and a series of computed measures derived from a required report, ETA 581. RQC is now concluding a period of voluntary implementation. New required reporting on the ETA 581 will become mandatory in the first quarter of Calendar Year 1995; the balance of the RQC will become mandatory effective January 1, 1996.

7. Action Required. SESA Administrators are requested to:

   1. Provide copies of this UIPL and Attachment to appropriate staff for review and comment.

   2. Forward comments to the National Office, with copies to the appropriate Regional Office, within 30 days from the date of this UIPL. Address comments and inquiries to:

      Mr. William N. Coyne
      PMR Project Officer
      U.S. Department of Labor
      Room S-4516 F.P.B., ATTN: TEUMC
      200 Constitution Avenue, N.W.
      Washington, D.C. 20210
      FAX Number: 202-219-8506
      

8. Attachment. "Performance Measurement Review Interim Evaluation Report" (Mathematica Policy Research, Inc., December 5, 1994).