DIRECTIVE	:	UNEMPLOYMENT INSURANCE PROGRAM LETTER NO. 20-91
TO	:	ALL STATE EMPLOYMENT SECURITY AGENCIES
FROM	:	DONALD J. KULICK Administrator for Regional Management
SUBJECT	:	Procedures for Release of Unemployment Insurance Benefits Quality Control (QC) Data

1. Purpose. To provide State employment security agencies (SESAs) with guidelines for the annual release of Unemployment Insurance (UI) Benefits Quality Control (BQC) program data for Calendar Year (CY) 1990.

2. References. 20 CFR 602; Federal Register Notice, March 26, 1990, 54 11112; andBenefits Quality Control State Operations Handbook, ET Handbook 395.

3. Background. The final rule to establish the Unemployment Insurance Quality Control program is found at 20 CFR Part 602. Effective October 5, 1987, QC programs in the 50 States, the District of Columbia, and Puerto Rico selected weekly random samples of UI benefit payments for investigation under the mandatory program.

   20 CFR 602.21(g) provides that each State shall:

   Release the results of the QC program at the same time each year, providing calendar year results using a standardized format to present the data as prescribed by the Department; States will have the opportunity to release this information prior to any release by the Department.

   The notice of procedures for the release of benefits QC data for the UI program was published in the Federal Register on March 26, 1990, at 54 FR 11112.

4. Date of Record. The Department has established close of business (c.o.b.) April 26, 1991, as the date of record for State databases for computation of the error rates for the annual report of QC data. All information in the annual report will be based on CY 1990 QC cases that have been signed off by the QC supervisor by c.o.b. April 26, 1991. States must not update any CY 1990 QC cases after this date until they have reviewed the computations of the error rates and have reconciled any differences. Any data analysis supporting the annual report must be run on the database as it exists at c.o.b. on April 26. If States make any further changes, the data would differ from the data that the National Office (NO) will use to compute final results for CY 1990 and any reconciliation cannot be accomplished.

5. Annual Report Software. The current software for the annual report will not change in regard to the computation of rates. The only changes are minor wording changes to the footnotes. States may run the rates as soon as all cases are completed. The date of record will be c.o.b. April 26, 1991.

   The NO will produce an annual report for each State, based on the QC cases as of c.o.b. April 26, 1991.

6. Annual Report Format. The annual report format is shown in Attachment 1. It is divided into three main sections.

   Quality Control Data

   Total Dollars Paid in Population

   Total UI benefits paid to the population of UI claimants who comprise the sampling frames for all weeks in CY 1990 for which the State pulled a QC sample.

   Sample Size

   Total UI payments selected during CY 1990 (QC batches 9001 through 9052) and completed (QC supervisor sign-off) by c.o.b. April 26, 1991. This is the number of QC sample cases from which the rates and sampling errors (confidence intervals) are estimated.

   Proper Payments

   The weighted ratio estimate of total dollars properly paid to total dollars paid expressed as a percentage.

   Overpayments

   The weighted ratio estimate of total dollars overpaid to total dollars paid expressed as a percentage.

   Underpayments

   The weighted ratio estimate of total dollars underpaid to total dollars paid expressed as a percentage.

   95% Confidence Interval

   A confidence interval, expressed as +- a percentage, will be constructed for each of the three estimated rates. The actual rate is expected to lie within 95 percent of the intervals constructed from repeated samples of the same size and selected in the same manner as the QC sample.

   Footnotes

   The following footnotes will appear on State QC annual reports to describe certain conditions that affect the data:

   Footnote 1:

   Claimants failing to conduct required work search were given formal warnings and no overpayment was established. The proper payment rate would be lower and the overpayment rate would be higher if these cases were counted as erroneous payments.

   Condition: Any case with key week action code '14' in fields hi2, hj2, or hk2 of the CY 1990 QC database.

   Footnote 2:

   Percentages apply to less than a complete year of UI payments due to the State not pulling a sample for xx weeks.

   Condition: There is no comparison report for one or more QC batches 9001 through 9052. (The CY 1990 summary of State QC sample selection is provided in Attachment 2.)

   Footnote 3:

   The precision of the percentage estimates may be reduced because the State selected samples that were below the minimum prescribed levels for xx weeks.

   Condition: State selected one or more QC weekly samples below the minimum level prescribed in Benefits Quality Control State Operations Handbook, ET Handbook No. 395, chapter III, p. 27. (See CY 1990 summary of State QC sample selection in Attachment 2.)

   Footnote 4:

   Percentages based on data collection procedures that were not in accordance with QC methodology prescribed in ET Handbook No. 395.

   Condition: Based on QC monitoring conducted by the Department of Labor Regional Offices and National Office.

   Footnote 5:

   The State completed xx% of the cases within 90 days. The QC program standard is 95% completed within 90 days.

   Condition: State failed to meet case completion objectives established in Benefits Quality Control State Operations Handbook, ET Handbook No. 395, chapter VI, p. 6.

   Footnote 6:

   Although x.x% of the sample cases were not completed when this report was prepared, this did not exceed the QC program requirement that no more than 2% of the cases for the year remain incomplete.

   Condition: Based on the number of QC cases that were not completed (QC supervisor sign-off) by c.o.b. April 26, 1991, divided by the number of valid cases selected for QC weekly samples during CY 1990 (QC batches 9001 through 9052). Note: Reporting the number of sample cases that were not completed is a generally accepted procedure in reporting the results of a statistical survey.

   Narrative

   States are invited to supply a narrative analysis that will accompany their annual report. The Department will include this narrative in the digest of State annual reports that it will prepare (see section 10, below).

   The narrative may include, but is not limited to, a discussion of State laws and how they affect error rates, an explanation of circumstances that led to some degree of inaccuracy in the reported error rates, and proposed or actual corrective action strategies.

   Since all QC units have WordPerfect 4.2 on their Artecon System, the Department prefers that the narrative be submitted electronically using that system. Prepare the narrative using WordPerfect software as you would with any other document. When the narrative is complete, press 2> then <F11> to select the Text InOut option. Select option 1 (Save-UNIX Text File Format). Name the narrative using the following convention: NARR plus the extension of the State's FIPS Postal abbreviation. For example, Rhode Island's narrative should be identified as NARR.RI; Florida's as NARR.FL. This does NOT transmit the narrative to the National Office but simply saves the document in a format ready for transfer. Be sure to follow the transmission instructions below. Then, exit WordPerfect, saving the document using a different name. (This is the narrative in WordPerfect format.)

   If it is not possible to submit electronically, the narrative may be submitted on paper. Paper transmittals must be submitted on 8 12" x 11" white bond (at least 20# weight), using letter quality printing that can be successfully photocopied. The narrative must not exceed two single-spaced pages.

   Using either method of submittal, the following format (see Attachment 3) is required:

   Margins -- one inch all around

   Title -- "ANNUAL REPORT NARRATIVE" typed in capital letters, and centered

   Subtitle -- "STATE NAME" typed in capital letters, and centered on the line below the title

   Date -- "CY 1990" centered on line below the subtitle

   Font -- Courier 10

   Spacing -- The text should follow three lines below the subtitle and end with a maximum length of three lines above the page number. Use six lines per inch.

   State narratives must be received in the NO no later than June 7, 1991. To send the UNIX Text File to the NO, at the Artecon prompt, type:

   mail uis6 !suem < NARR.name

   Substitute your State document name, i.e., NARR.RI for the NARR.name in the above expression. Note also the positions of single spaces in the expression, i.e., between mail and uis6; between suem and <; between < and NARR.name. For help with this process, call the QC Hotline on 1-800-523-9031.

   Paper transmittals should be sent to the attention of Susan Makara at:

   U. S. Department of Labor
   200 Constitution Ave., NW
   Attn: TEUQS, Room S-4015
   Washington, DC 20210

   If the State does not plan to submit a narrative, the State should notify the NO at the above address or electronically by June 7, 1991, that no narrative will be submitted.

7. Supplemental Data. In order for the SESAs to interpret the data in the annual report and provide narrative explanations of the data, the Department recognizes the need for additional information on cause and responsibility. The software that produces the QC annual report will also generate a separate report consisting of up to 16 responsibility categories and up to 6 cause categories. If the categories are equal to "0" the item will not be displayed in the table. Responsibility and cause data will be reported as percentages of UI dollars overpaid. The format for this report is shown in Attachment 4.

   In order to provide States with the greatest flexibility in interpreting the data, all possible responsibility codes and broad cause categories can be generated. States can combine categories which contain few sample cases, where appropriate. States should be aware that many of these percentages are based on small numbers of QC sample cases and may have relatively large sampling errors.

8. Comment Period. QC annual reports produced by the Department of Labor will be transmitted to the States on May 10, 1991. Each State should review its annual report and if there are any comments submit them to the NO (Attn: TEUQS, Andy Spisak by May 31, 1991). SESA comments will be discussed and reconciled by June 7, 1991.

9. Public Release by SESA. As stated in the Federal Register notice of March 26, 1990, (54 FR 11112):

   Each State will release the required data annually through established channels for disseminating State performance data. In accordance with established State procedures, the data must be provided to those who normally receive performanceevaluation data and to anyone else who requests the data. It is no longer necessary to publish a notice of availability.

   States must release their annual reports by June 28, 1991. If a State fails to release data in accordance with these procedures, the Department, in its annual review of State QC operations as specified in 20 CFR 602.31, will take appropriate action.

10. Federal Release of Data. The Department will announce the availability of QC program data for CY 1990 through a notice published in the Federal Register on or about July 31, 1991. The Department's publication will be a digest prefaced by background information on the Quality Control program, and a brief discussion of how the error rate estimates were computed. Readers will be discouraged from comparing results among States. Each State's section, including its annual report and narrative, will be displayed separately.

    For CY 1990, the Department will continue the practice begun with CY 1989 data of publishing certain analytical information on aspects of the QC findings. After consultations with the Washington based stake holders in the UI system, it was decided to provide additional information in the QC annual report. Specifically there will be an analysis of the national under payments, individual State cause and responsibility information, and trend charts on over payments for each State in volume II. The frequency distribution data contained in volume II of the CY 1989 report will be continued in this report. State sections will be ordered alphabetically to discourage ranking and comparisons.

    If the Department determines that a State has failed to follow the prescribed QC methodology--in terms of sample selection, data collection methodology, or case completion timeliness--to such an extent that the data are not reliable, the Department will publish an explanation of the major deficiency in lieu of an annual report for that State. Also in accordance with instructions in the Program and Budget Plan, ET Handbook No. 336 (2nd edition), States will be required to address these areas by submitting corrective action plans for the upcoming fiscal year.

11. Key Dates. The following is a summary of key dates for the public release of CY 1990 QC data.

    April 26	Update the annual report software electronically (DOL) to include footnote modifications
    April 26	Establish date of record for State QC databases (DOL)
    May 1-10	Generate annual reports for all States (DOL)
    May 10	Send annual reports to the States (DOL)
    May 13-31	Review (States) annual report
    June 3-7	Discuss and reconcile comments with DOL (States)
    June 7	Provide information on contact person and SESA Narratives due DOL (Note: State notification to DOL that no narrative will be submitted is requested)
    June 28	Release CY 1990 QC annual report (States)
    July 31	Publish notice in the Federal Register announcing the availability of QC data (DOL)

12. Action Required. State Administrators are requested to:

    1. provide copies of these guidelines to the appropriate staff; and

    2. ensure that the following information is supplied to ETA Regional Administrators:

       1. the name, location, and telephone number of a contact person who interested parties may write or call for State specific interpretations by June 7, 1991;

       2. a copy of the State Annual QC data release by June 28, 1991; and

       3. copies of any press releases and or newspaper articles relating to the annual report released by the SESA for CY 1990 by July 5, 1991.

13. Inquiries. Questions should be directed to the appropriate Regional Office.

14. Attachments. (1) Annual Report Format; (2) State QC Sample Selection by Batch for CY 1990; (3) Format for QC Annual Report Narrative; and (4) Format for QC Annual Report Supplemental Data