Practical significance is a conceptual framework for evaluating discrimination cases developed primarily on statistical evidence that is the subject of increasing interest and discussion by some in the equal employment opportunity (EEO) field.
- What is practical significance?
- How is practical significance related to statistical significance?
- How is practical significance measured?
- Do the Uniform Guidelines on Employee Selection Procedures address practical significance?
- Does OFCCP consider practical significance during a compliance review?
What is practical significance?
In the EEO context, practical significance refers to whether an observed disparity in employment opportunities or outcomes reflects meaningful harm to the disfavored group. The concept focuses on the contextual impact or importance of the disparity rather than its likelihood of occurring by chance.
How is practical significance related to statistical significance?
Since the "importance" of a disparity is influenced by the magnitude of the impact, the notions of practical and statistical significance are related. Statistical significance is a function of multiple factors, including the magnitude of the disparity, the number of observations in the analysis, and the power of the statistical test used. The purpose of a statistical test is to assess the likelihood that random or legitimate, nondiscriminatory factors rather than discriminatory factors produced an observed disparity. Under certain conditions, a small disparity may be statistically significant due in large part to the size of the data set. OFCCP is mindful that enforcement efforts to eliminate small impacts may prove counterproductive due to the difficulty of an alternative practice improving upon an already small effect.
Conversely, just because a disparity is not statistically significant does not necessarily mean that discrimination did not occur. Discrimination cases need not be founded upon a statistically significant disparity. A disparity that does not meet the usual statistical significance standard may nevertheless be compelling if, for example, there is a glaring selection shortfall supplemented with strong anecdotal evidence, or red flags in the company’s EEO compliance.
How is practical significance measured?
There is no universally accepted measure of practical significance in the EEO field. What is considered practically significant depends on the employment opportunity at issue and the specific facts of the case. There are a variety of measures and techniques discussed in the literature and case law, but no superior or consistently reliable measure has emerged. Some of the measures used in employment discrimination cases include the impact ratio, the odds ratio, the flip-flop rule, the Apsley v. Boeing ratio, the standardized difference between disfavored and favored outcomes, and Cohen’s h.
Do the Uniform Guidelines on Employee Selection Procedures address practical significance?
Yes. The Uniform Guidelines on Employee Selection Procedures (UGESP), codified at 41 CFR part 60-3, contemplate practical significance in Section 4D. "Smaller differences in selection rates [i.e., not meeting the four-fifths rule], may nevertheless constitute adverse impact, where they are significant in both statistical and practical terms."
In addition, the Adoption of Questions and Answers to Clarify and Provide a Common Interpretation of the Uniform Guidelines on Employee Selection Procedures (UGESP Q&A, 44 Fed. Reg. 11996 (Mar. 2, 1979))restates and expands on Section 4D with specific examples.
- Q&A 20 provides two hypothetical examples to illustrate how a final determination of adverse impact may be made contrary to the four-fifths rule based on practical significance considerations.
- In the first example, arrest records disqualified 10% of Hispanic applicants and 4% of non-Hispanic White applicants. A disfavored group disqualification rate 2.5 times that of the favored group was considered "large enough to be practically significant," even though the impact ratio was 0.94.
- In the second example, four hires from an applicant pool of 30 was considered "too small to warrant a determination of adverse impact," even though the impact ratio was 0.67.
- Q&A 21 provides what has been referred to as the "flip-flop" or "N of 1" rule, stating, "Generally, it is inappropriate to require validity evidence or to take enforcement action where the number of persons and the difference in selection rates are so small the selection of one different person for one job would shift the result from adverse impact against one group to a situation in which that group has a higher selection rate than the other group." However, note that even if a disparity were determined not to be practically significant by this rule, adverse impact might still be inferred by OFCCP if it "continued over a period of time, so as to constitute a pattern."
Does OFCCP consider practical significance during a compliance review?
Yes. OFCCP is required to consider practical significance as part of a holistic evaluation in accordance with OFCCP’s recently codified final rule, Nondiscrimination Obligations of Federal Contractors and Subcontractors: Procedures To Resolve Potential Employment Discrimination (“final rule”), along with statistical significance and all other evidence gathered in the course of the investigation. See, e.g., 41 CFR § 60-1.33(a)(1)(ii). Depending on the employment issue under review, OFCCP utilizes a variety of practical significance measures discussed herein and in the referenced materials. Although the agency may consider multiple practical significance measures, the preamble to the final rule on resolution procedures includes guidance on practical significance to provide further clarity on the topic. See 85 Fed. Reg. 71559-60. Some of the measures of practical significance that have been used in employment selection cases include the size of the selection shortfall, the “four-fifths rule” (or “80 percent rule”), the impact ratio, and the odds ratio. In the selection context, OFCCP will typically use the following impact ratio thresholds to assess practical significance and determine whether OFCCP will pursue enforcement:
Impact Ratio of Selection Rates
> 0.9 Very Unlikely
0.8 - 0.9 Unlikely
0.7 - 0.8 Likely
< 0.7 Very Likely
An impact ratio of 0.8 is a frequently cited benchmark in the equal employment opportunity literature for determining whether the impact ratio of a selection disparity is practically significant, which is why OFCCP has chosen it as the hinge point between a likely and unlikely finding of practical significance for selection decisions. For impact ratios below 0.9, OFCCP will apply its discretion in determining whether to issue a pre-enforcement notice according to the strength or weakness of the evidence in particular cases, but the agency will require strong additional supporting evidence when the impact ratio is between 0.8 and 0.9. In addition, because the impact ratio is a less effective statistical measure when selection rates are very small, OFCCP utilizes a 3% disparity between the selection rates of disfavored and favored groups as a general minimum threshold for a finding of practical significance, although there may be situations with very low selection rates, such as a 4% selection rate for the favored group and a 1% selection rate for the disfavored group, where the odds ratio and other evidence would still support a finding of practical significance.
In the compensation context, multiple measures of practical significance are also viable. Some of the most common methods are the percentage of pay disparity, the standardized difference between disfavored and favored groups, and the more technical Type II squared semi-partial correlation coefficient. The preamble to the final rule states that OFCCP’s standard measure of practical significance will be the percentage difference in compensation, which refers to the percentage difference between the mean compensation of employees within the disfavored group in proportion to the mean compensation of employees within the favored group. For this measure, OFCCP will use the following thresholds to assess the likelihood that OFCCP will find practical significance in the compensation context:
Size of Compensation Disparity
< 1% Very Unlikely
1 - 2% Unlikely
2 - 5% Likely
> 5% Very Likely
For compensation disparities above 1%, the agency has discretion in determining whether to issue a pre-enforcement notice according to the facts and circumstances of individual cases, but OFCCP will be unlikely to determine that a compensation disparity below 2% is practically significant unless there is additional strong supporting evidence. When compensation disparities are greater than 5%, OFCCP will nearly always find that a compensation disparity is practically significant if the agency also determines that its statistical model is sound. In rare cases, OFCCP may also apply more rigorous practical significance tests to measure the import of compensation disparities, such as the standardized difference between disfavored and favored groups or the Type II squared semi-partial correlation, which help ensure the agency is applying its practical significance standard relatively uniformly across administrative cases.
OFCCP will use the measures above to make an informed decision on the potential strength of the case and whether, in light of the quantitative and qualitative evidence, the size of an observed disparity justifies moving forward with enforcement procedures.
The contents of this document do not have the force and effect of law and are not meant to bind the public in any way. This document is intended only to provide clarity to the public regarding existing requirements under the law or agency policies.
Last updated on January 15, 2021