DIRECTIVE	:	UNEMPLOYMENT INSURANCE PROGRAM LETTER NO. 22-99
TO	:	ALL STATE EMPLOYMENT SECURITY AGENCIES
FROM	:	GRACE A. KILBANE Director Unemployment Insurance Service
SUBJECT	:	Procedures for Release of Unemployment Insurance (UI) Benefit Accuracy Methodology (BAM) Data for Calendar Year (CY) 1998

1. Purpose. To provide SESAs with guidelines for the annual release of UI BAM program data for CY 1998.

2. References. 20 CFR 602; Federal Register Notice, September 3, 1987, 52 FR 33520; Unemployment Insurance Program Letter (UIPL) 13-93, January 12, 1993; UIPL 19-96, April 24, 1996; and Benefit Accuracy Measurement State Operations Handbook , ET Handbook No. 395.

3. Background. The final rule to establish the UI Benefit Accuracy Measurement, formerly Benefits Quality Control (QC), program is found at 20 CFR Part 602 (52 FR 33520). 20 CFR 602.21(g) provides that each State shall:

   Release the results of the QC program at the same time each year, providing calendar year results using a standardized format to present the data as prescribed by the Department of Labor; States will have the opportunity to release this information prior to any release by the Department of Labor.

   UIPL 19-96 announced the elimination of the requirement that States publicly release BAM program data. While the Department of Labor will continue to publish an annual compendium of all States' data, each State can determine whether to independently release its data to the public.

4. Federal Release of Data.

   UI PERFORMS Data

   BAM data are published as part of a UI PERFORMS report that also includes data for the Benefit Timeliness and Quality (BTQ) and Tax Performance System (TPS) programs. UI PERFORMS is the Department of Labor's management system for promoting continuous improvement in the performance of the UI system.

   The CY 1998 UI PERFORMS Annual Report, including State narratives, will be posted on the Information Technology Support Center (ITSC) Internet site (www.itsc.state.md.us). Printed versions of the report will be available through the National Technical Information Service (NTIS). The printed version will not include State narratives. The Department of Labor will announce the availability of CY 1998 BAM program data and information on ordering the CY 1998 UI PERFORMS Annual Report from the NTIS in a notice published in the Federal Register on or before July 30, 1999.

   The BAM section of the CY 1998 UI PERFORMS Annual Report will include background information on the BAM program and a brief discussion of how the payment accuracy rate estimates are computed. Readers will be discouraged from comparing results among States. Each State's BAM Annual Report will be displayed separately in alphabetical order to discourage ranking and comparisons. In addition to the States' Annual Reports, the Department of Labor may publish analytical information on CY 1998 BAM program findings.

   If the Department of Labor determines that a State has failed to follow the prescribed BAM program methodology -- regarding sample selection, data collection methodology, or case completion timeliness -- to such an extent that the data are not reliable, the Department of Labor will publish an explanation of the major deficiency in lieu of a BAM Annual Report for that State. States will be required to address these areas by developing corrective action plans as part of the State Quality Service Plan process.

   Narrative

   States may supply an optional narrative analysis to accompany the publication of UI PERFORMS data. The Department of Labor will include this narrative as part of the UI PERFORMS data that will be posted on the ITSC Internet site. However, State narratives will not be published in the UI PERFORMS CY 1998 Annual Report.

   The narrative may address any of the data published in the UI PERFORMS Annual Report, including BAM payment accuracy rates, Benefit Timeliness and Quality data, and Tax Performance System data. Narratives may include a discussion of State laws or special circumstances that affected the UI PERFORMS data, and continuous improvement plans that have been proposed or adopted.

   The narrative must be transmitted by electronic mail to Yvette Sasseen using "tonight!yvette" on the Sun SPARC computer system or to "ysasseen@doleta.gov" on the Internet. List the subject as "UI PERFORMS Narrative for XX" where XX is the State Postal FIPS abbreviation. For help with this process, call the Division of Information Technology Hotline on 1-800-473-0188. Any narrative submitted other than electronically will not be included in the UI PERFORMS data posted on the ITSC Internet site.

   State narratives must be received in the National Office(N0) no later than June 4, 1999. States that decide not to submit a narrative should notify the National Office no later than June 4, 1999, that no narrative will be submitted. States will also provide the appropriate Regional Office (RO) a copy of their narrative or notice that no narrative will be submitted. State narratives received after the June 4, 1999, deadline will not be included in the UI PERFORMS data posted on the ITSC Internet site.

5. BAM Date of Record. The Department of Labor has established close of business (c.o.b.) April 30, 1999, as the date of record for State databases for computation of the payment accuracy rates for the CY 1998 Annual Report of BAM data. All information in the Annual Report will be based on CY 1998 BAM cases that have been signed off by the BAM program supervisor by c.o.b. April 30, 1999. Any data analysis supporting the Annual Report must be run on the database as it exists at c.o.b. on April 30, 1999.

   States must not reopen any CY 1998 BAM cases after this date until they have reviewed the computations of the error rates and have reconciled any differences between the State's and Department of Labor's databases. If States make any subsequent changes, their data will differ from the Department of Labor's data, which can result in inconsistencies between the State and Department of Labor versions of the Annual Report. Any such difference will have to be reconciled before the Annual Report can be issued.

6. BAM Annual Report Software. As described in UIPL 13-93, the software which produces the BAM Annual Report excludes cases which do not meet the definition of the BAM population, for example, interstate or supplemental payments. These cases are coded "9" in the program code data element (c1 in the b_master table of the UI database). The BAM Annual Report software adjusts the amount of UI benefits paid to the population to exclude the amount paid that is represented by cases with program code value "9".

   Prior to the date of record, the Department of Labor will transmit a footnote lookup table for the CY 1998 BAM Annual Report. States can run the Annual Report software as soon as all BAM cases are completed. The NO will produce an Annual Report for each State, based on the CY 1998 BAM cases as of c.o.b. April 30, 1999.

7. BAM Annual Report Format and Definitions.  The BAM Annual Report format is shown in Attachment 1. The components of the Annual Report are define in this section.

   Benefit Payment Accuracy Data

   Total Dollars Paid in Population

   Total UI benefits paid to the population of UI claimants who comprise the sampling frames for all weeks in CY 1998 for which the State pulled a BAM sample, adjusted to exclude other UI payments that do not meet the definition of the BAM population.

   Sample Size

   Total UI payments selected during CY 1998 (BAM batches 9801 through 9852) and completed (supervisor sign-off) by c.o.b. April 30, 1999, excluding cases that do not meet the BAM population definition. This is the number of BAM sample cases from which the payment accuracy rates and confidence intervals are estimated.

   Proper Payments

   The weighted ratio estimate of total dollars properly paid to total dollars paid expressed as a percentage.

   Overpayments

   The weighted ratio estimate of total dollars overpaid to total dollars paid expressed as a percentage.

   Underpayments

   The weighted ratio estimate of total dollars underpaid to total dollars paid expressed as a percentage.

   95 Percent Confidence Interval

   A confidence interval, expressed as +/- a percentage, is constructed for each of the three estimated rates. The actual rate is expected to lie within 95 percent of the intervals constructed from repeated samples of the same size and selected in the same manner as the BAM sample.

   Footnotes

   The footnotes that appear on State BAM Annual Reports describe certain conditions that affect the data. These footnotes are defined in Attachment 1.

8. Supplemental Data.   In order for the SESAs to interpret the data in the BAM Annual Report, the software that produces the BAM Annual Report also generates a separate report consisting of up to 16 responsibility categories and up to 6 cause categories. Responsibility and cause data are reported as percentages of UI dollars overpaid. No sampling errors are computed for these percentages. States should be aware that some of these percentages are based on small numbers of BAM sample cases and may have relatively large sampling errors. States can combine categories which contain few sample cases, where appropriate. The format for this report is shown in Attachment 2.

   If cause and responsibility percentages do not add up to 100 percent, States should check for cases in which the key week amount overpaid coded in data element h5 in the b_master table of the UI database does not equal the sum of dollars overpaid coded in data element ei1 for key week actions 10, 11, 12, 13, and 15 in the b_errisu table. The amount overpaid cannot exceed the amount paid coded in data element f13 in the b_master table.

9. Comment Period. Department of Labor will be mailed to each State by May 10, 1999. Each State should carefully review its Annual Report and submit any comments to the NO by May 28, 1999, to:

   Andrew Spisak
   U.S. Department of Labor
   ETA/Unemployment Insurance Service
   200 Constitution Avenue, NW
   Room S-4522
   Washington, DC 20210

   Alternatively, States may submit comments by e-mail to aspisak@doleta.gov via the internet or tonight!andy via the State's Sun computer e-mail utility or by fax to Andrew Spisak at (202) 219-8506.

   Each State will send a copy of any comments sent to the NO to the appropriate RO by May 28, 1999. National and RO staffs will discuss and reconcile issues raised in the States' comments by June 4, 1999.

10. Public Release by SESA.   Although States are no longer required to release their annual BAM program data, States should provide the data to interested parties upon request. The name of the State official designated by the State as its BAM Annual Report contact will be published in the Federal Register notice announcing the availability of BAM data in the UI PERFORMS Annual Report. States must continue to follow the reporting requirements described in section 11, regardless of their decision to release their data to the public.

11. Key Dates. The following is a summary of key dates for the public release of CY 1998 BAM data. These dates are the latest that the task or action must be completed.

    4/23  States submit requests for waivers of BAM time lapse requirements, along with supporting analysis, to the appropriate Department of Labor ETA Regional Administrator.

    4/30  The Department of Labor RO respond to States on the disposition of State requests for waivers of BAM time lapse requirements.

    4/30  The Department of Labor electronically transmits the footnote look-up table for the BAM Annual Report software to each State's Sun computer.

    4/30  Each State checks its UI database to: 1) ensure that all cases for which the BAM investigations have been completed have been signed off by the BAM supervisor, and 2) reopen any case in order to update information or correct any data errors.

    Cases closed by supervisors after April 30, 1999, will not be included in the Annual Report. States should not reopen any CY 1998 cases to change the database after April 30, 1999, until their Annual Report is finalized.

    5/10  Grace A. Kilbane, Director, UIS, sends the BAM Annual Reports to the State Administrators.

    5/28  States review Annual Reports, including footnotes, and provide comments to the NO, with copies to ETA Regional Administrators.

    6/4  National and RO discuss SESA comments with the States and reconcile any differences between the State's and Department of Labor's databases.

    6/4  States electronically transmit to the NO, with a copy to the ETA Regional Administrators, narratives or notification that a narrative will not be submitted.

    6/4  States provide name, address, and phone number of BAM Annual Report contact person (or persons) to ETA Regional Administrators.

    7/30  The Department of Labor publishes notice in the Federal Register announcing the availability of BAM data as part of the UI PERFORMS Annual Report.

12. Action Required. State Administrators are requested to:

    1. provide copies of these guidelines to the appropriate staff;

    2. ensure that the following items are supplied to ETA Regional Administrators:

       (1)  any request for a waiver of time lapse requirements by c.o.b. April 23, 1999; and

       (2)  the name, address, and telephone number of the contact person(s) to whom interested parties may write or call with questions or inquiries about the State's BAM program data by c.o.b. June 4, 1999.

    3. ensure that the following items are supplied to the Department of Labor's NO with copies sent to ETA Regional Administrators:

       (1)  any comments on the CY 1998 Annual Report data that require reconciliation of differences between the State's and Department of Labor's databases by c.o.b. May 28, 1999; and

       (2)  a narrative or notification that a narrative will not be submitted by June 4, 1999.

13. Inquiries. Questions should be directed to the appropriate RO.

14. Attachments.

    (1) BAM Annual Report Format and Footnotes and

    (2) Format for BAM Annual Report Supplemental Data.

Attachment 1

BAM ANNUAL REPORT
(New Version)
CY 1998

Run as of MM-DD-YY

State: ZZ

Unemployment Insurance Benefit Accuracy Measurement
Annual Report for Calendar Year 1998

Total Dollars Paid in Population		$x,xxx,xxx,xxx
Sample Size		x,xxx
	Percentage of Dollars	95% Confidence Interval
Proper Payments	xx.x%	xx.x%
Overpayments	xx.x%	xx.x%
	100.0%
Underpayments	xx.x%	xx.x%

Footnotes

An explanation of the footnotes follows (on the next two pages).

BAM ANNUAL REPORT FOOTNOTES

Claimants failing to conduct required work search were given formal warnings and no overpayment was established. The proper payment rate would be lower and the overpayment rate would be higher if these cases were counted as erroneous payments.

Condition: Any CY 1998 BAM case with key week action code '14' in field ei2 of the b_errisu table of the UI database.

Footnote 2

Percentages apply to less than a complete year of UI payments due to the State not pulling a sample for xx weeks.

Condition: There is no record in the b_comparison table of the UI database for one or more BAM batches 9801 through 9852.

Footnote 3

The State selected samples that were below the minimum prescribed levels for xx weeks.

Condition: State selected one or more BAM weekly samples below the minimum level prescribed in Benefit Accuracy Measurement State Operations Handbook, ET Handbook No. 395, chapter III, p.27. Cases sampled which fail to meet the BAM population definition are counted toward meeting the minimum weekly sample.

Footnote 4

Percentages based on data collection procedures that were not completely in accordance with the program methodology prescribed in ET Handbook No. 395.

Condition: Based on program monitoring conducted by the Department of Labor ROs and Nos.

Footnote 5

The State completed xx percent of the cases within 90 days. The program standard is 95 percent completed within 90 days.

Condition: State failed to meet case completion objectives established in Benefit Accuracy Measurement State Operations Handbook, ET Handbook No. 395, chapter VI, p. 6. Cases not meeting the BAM population definition are not counted in calculating State time lapse rates.

Footnote 6

x.x percent of the sample cases were not completed when this report was prepared. This exceeded the program requirement that no more than 2 percent of the cases for the year remain incomplete.

Condition: The percentage is based on the number of BAM cases that were not completed (no supervisor sign-off) by c.o.b. April 30, 1999, divided by the number of valid cases selected for BAM weekly samples during CY 1998 (BAM batches 9801 through 9852). Cases that do not meet the BAM population definition are not counted in calculating State case completion rates.

Footnote 7

The annual sample for [State] is [#] cases below the allocated annual sample for the State. The precision of the data might be reduced due to the failure to sample at the prescribed level.

Condition: The State's annual sample selection is 60 or more cases below its allocated annual sample (with alternative data collection methods, one staff year equals 120 cases). Cases that do not meet the BAM population definition are counted toward meeting the annual sample allocation.

Footnote 8

The population from which the BAM sample was selected did not include all of the UI benefits paid. This limits the degree to which inferences about the population can be made from BAM data.

Condition: The dollars paid in the BAM population for CY 1998 vary from the dollars paid reported in the State's ETA 5159 Claims and Payment Activities reports by more than the statistical control limit.

Attachment 2

BAM ANNUAL REPORT
Supplemental Data
CY 1998

Responsibility for Overpayments (Percent of Dollars Overpaid)

Claimant Only	xx.x
Employer Only	xx.x
Agency Only	xx.x
Claimant + Employer	xx.x
Claimant + Agency	xx.x
Employer + Agency	xx.x
Claimant + Employer + Agency	xx.x
Other Only	xx.x
Claimant + Other	xx.x
Employer + Other	xx.x
Agency + Other	xx.x
Claimant + Employer + Other	xx.x
Claimant + Agency + Other	xx.x
Employer + Agency + Other	xx.x
Claimant + Empl. + Agy. + Oth.	xx.x
Responsibility Not Specified	xx.x

Cause of Overpayments (Percent of Dollars Overpaid)

NOTE:  The percentages for the responsibilities or causes do not sum to 100 percent. You should check field (h5) to verify that all cases have been coded correctly within your specified batch range. Also the responsibilities and cause categories are ranked by their respective percents.