2M Employment Activity Data Requirements

a. Identifying and Mapping the Employment Process. The CO must determine how the contractor’s selection process for the job or opportunity under examination works, including identifying all of the steps and decision points involved. The employment process usually consists of decisions and actions an employee or applicant must take such as applying for the job, as well as qualification points such as passing a test or interview in order to move to the next stage of the process. Once the CO identifies the employer’s process, he or she may find it helpful to “map” or create a diagram representing the process and the points or steps in the process where the contractor can eliminate an employee or applicant from further consideration. In addition to mapping the process, the CO will need to gather information to understand when the process was instituted, why, by whom, whether there have been changes in the process and, if so, when and why.

b. Identifying Pass/Fail Points. If an individual must successfully navigate a certain step(s) in the process before proceeding on to the next, that step(s) is a “pass/fail point.” Ideally, the CO can examine each pass/fail point to see if the contractor is disproportionately screening out members of any specific group. However, if the contractor does not eliminate individuals at separate steps throughout the process, but instead weighs their performance at each step to formulate a final decision, then the CO evaluates the process as a whole. To do this, the CO must ask the contractor to explain the importance and weight of each step in the selection decision. If an analysis of the entire selection process shows that the contractor disproportionately screens out members of a specific group, the CO must attempt to determine the particular criteria that are the source of the problem.

c. Lack of Data: If the contractor failed to maintain adequate data as required by 41 CFR Part 60-3, the CO must cite the contractor for this violation. If the contractor’s lack of data hinders the CO’s ability to determine the step or criterion that disproportionately eliminated members of the nonfavored group, then the CO must conduct the analysis based upon the “bottom line effect” or the existence of an adverse impact due to the total selection process. However, under such circumstances, the CO must obtain as much information as possible about the process and make every effort to identify the step or criterion causing the problem. Below is a very basic example of the concept.

Example. A selection process consists of: 1) completing an application; 2) having a high school diploma; 3) taking a written test; and 4) participating in an interview during which the contractor accepts or rejects the applicant. In this example, applicants must demonstrate they have a high school diploma before taking the test, and must pass the test before participating in an interview. For each applicant, the CO must determine the applicant’s race, ethnicity and sex, whether the applicant had a high school diploma, the test results, and the results of the interview including the interviewer’s notes, if any. This information is used when determining the number of members within each group eliminated at each step of the process. The CO must conduct an adverse impact analysis at each stage of the selection process to determine whether adverse impact occurs.

If the contractor’s application process, or any part of the process, involves Internet applicants, the CO must obtain applicant data and information regarding the basic criteria used in determining applicants for the job.164 The CO must discuss with his or her supervisor the appropriate analyses for Internet applicants.

d. Relevant Pools. The relevant pool for each step of the contractor’s selection process includes the individuals who made it to that step in the selection process compared with those who made it through to the next step. In the example in (c), above, if the CO is analyzing disparity in hiring rates, he or she would first consider the pool of all applicants to determine whether the high school diploma requirement is the source of the adverse impact. If the high school diploma requirement does not fully explain the disparity in hiring, the CO would next determine whether the test disproportionately eliminated members of the nonfavored group. Finally, the relevant pool for evaluating the effect of the interview would be all applicants who have a high school diploma and passed the test. Therefore, any difference in their selection rates would be attributable to the interview.

The contractor’s selection criteria fall into two broad categories: “objective criteria” and “subjective criteria.”

a. Objective Criteria. The central characteristic of an objective criterion is that it can be independently verified. With objective criteria, different people measuring the criteria will reach the same results because they are clearly defined or quantifiable in nature. For example, whether or not the individual earned a certain diploma or degree is an independently verifiable fact.

b. Subjective Criteria. Subjective criteria require judgment in their application. Therefore, people can differ in opinion on whether a particular candidate possesses and meets such a criterion. For example, two selecting officials may easily have differing opinions on whether a candidate “has good leadership skills.” In this example, if disparities exist in the application of the subjective criteria between groups, the CO would gather information to determine if the contractor applied the subjective criteria based on bias or stereotypes. An example of this might be a statement made by hiring official that men commonly display leadership qualities that women do not.

In most cases, the CO will first analyze a contractor’s use of objective criteria and the resulting impact, followed by an analysis of the contractor’s use of subjective criteria and the resulting impact. Decisions based on subjective criteria involve the use of “judgment calls” by the contractor. Clearly, it can be difficult to assess whether subjective criteria are applied equally to all similarly situated applicants or candidates, or whether the application of subjective criteria is tainted by bias or stereotyping. Generally, when the CO cannot attribute an adverse impact to objective criteria, it is likely attributable to the use of subjective criteria. This situation is most likely where the CO finds that the reasons for the decision are undocumented or the contractor characterizes the decision in a subjective manner like saying that the applicant was “a good fit.”

In some instances, the CO will find that the contractor used multiple criteria when making the hiring decision. When this occurs, the CO must determine if the person selected was required to meet each criterion individually or whether the contractor balanced the criteria (e.g., assigns weight to each criterion) in reaching a decision. If meeting each criterion was a requirement, the CO must perform the appropriate analysis for each one. If the criteria are weighted, the CO must establish how they were weighted in order to determine how to conduct an analysis.

For example, a contractor gives each criterion a specific weight but one criterion is not met by a candidate. The contractor, however, selects that candidate. The CO must analyze each criterion independently to, among other things, determine the relevance of each criterion in the selection process. If, however, a contractor gives each criterion a score, and it is the cumulative score that determines whether a candidate is selected, the CO would analyze the cumulative criteria. The contractor may also refer to criteria as “minimum” or “preferred.” Minimum qualifications are usually treated as “pass/fail” whereas preferred qualifications are those that the contractor may prefer but are not necessarily required. The CO must determine whether, and how, the contractor is using this criterion in making employment decisions.

The first step in analyzing the contractor’s use of objective selection criteria is to verify that the criteria were actually used to make selections, and that they were applied uniformly to all members of the favored and nonfavored groups. COs only need to consider the contractor’s stated legitimate objective criteria.

a. Single Criterion. To examine the application of the criterion, the CO must determine whether the same criterion was applied to each individual and that data reflecting the application of the criterion are available for all individuals in the relevant pool.

b. Multiple Criteria. When there are multiple criteria, or there is no summary record available to determine whether the people selected met the criteria, the CO will need to create a summary record. The CO can do this by creating a spreadsheet or a database that contains all relevant data for each individual applicant or candidate, or both. Based on the example in subsection 2M01(a) above, the spreadsheet data for each applicant would include race, ethnicity, sex, high school diploma or no diploma, test result and interview result. The spreadsheet will allow the CO to analyze the data more readily and determine whether the multiple criteria were applied uniformly to each applicant and the result of their application (e.g., the applicant or candidate was either selected or not selected).

c. Interpreting the Results of Criteria Verification.

• When the review of individuals selected is complete, the CO must review the spreadsheet data to see if each selectee actually met the criteria. The CO may conclude that the contractor used the criteria if everyone selected met the criteria.
• If a substantial number of individuals selected do not meet a particular criterion, the CO may discount the contractor’s statement that the criterion was the basis for the decision.
• If the contractor used the criterion in a nonuniform manner, the CO must determine whether the contractor applied the criterion differently to members of the favored group when compared to the nonfavored group. This comparison will reveal any disparate treatment.
• If the contractor applied the criterion uniformly to members of the nonfavored group and favored group, then the CO must analyze the effect of the application of the criterion for adverse impact.

a. Pool of Individuals Used for Statistical Calculation. In an adverse impact calculation, the CO compares the number of individuals from each group (by race, ethnicity, and sex) who were assessed using the criterion at issue with the number from each group who met the criterion. This yields a “pass rate” or “selection rate” for each group, and the CO then tests the difference in pass rates between the groups for adverse impact.

b. Performing the Adverse Impact Calculation. First, the CO must determine whether each objective criterion is a pass/fail requirement, (i.e., it eliminates an individual from further consideration if it is not met), or whether it is considered along with other criteria in making a final decision.

• For each pass/fail criterion, the CO must determine whether the difference in pass rates for members of each group (by race, ethnicity, and sex) shows adverse impact.
• When the criteria are not considered singly but are considered together in reaching a decision, the CO must determine how the contractor weighted each criterion to make a selection. Using the weights used by the contractor, the CO must determine whether this is causing the adverse impact. If it is not possible to isolate the contribution of the individual criterion to the bottom line effect, the CO will calculate the cumulative effect of all the criteria. This calculation may be appropriate when the contractor did not keep records of the effects of the individual criteria.

c. Measurement of Statistical Significance. Measurement of the statistical significance of a difference in selection rates, whether by standard deviations or another method, indicates the probability that a particular disparity in those rates could or could not have occurred by chance. The greater the disparity between the percentage of a specific group (e.g., women) actually selected and the percentage of the group expected, the more likely the variation did not happen by chance but was a result of the selection factor used to screen. When there is a significant probability that the disparity in selection rates for members of a nonfavored group could not have happened by chance, this is strong evidence of adverse impact.

The CO will generally undertake an analysis of subjective criteria only after first determining the effects of any objective qualifications used by the contractor. Subjective criteria are analyzed in essentially the same way as objective criteria: namely, the CO first makes a determination as to whether the contractor is actually using the stated subjective criteria (verification) and then whether the contractor is applying them to members of all groups (if not, disparate treatment may have occurred). If the contractor applied subjective criteria to all groups, then use of subjective criteria will be analyzed for adverse impact.

The CO determines this by comparing the selection rates of members of favored and nonfavored groups who met the subjective requirements to determine whether statistically significant differences exist. If the disparity in these selection rates is not significant, then it is highly unlikely that discrimination resulted from the use of the subjective criteria. If the disparity is significant, then adverse impact exists and disparate impact discrimination may have occurred. The adverse impact analysis here is the same as the analysis described in 2M03 for the objective criteria analysis.

According to the UGESP, the use of any selection procedure or criterion that has an adverse impact on hiring, promoting or other employment opportunities of members of any race, ethnic group, or sex is discriminatory unless the procedure is shown to be validated or otherwise job-related and consistent with business necessity.165

It is the contractor’s responsibility to provide validity evidence sufficient to justify each selection procedure or criterion found to have adverse impact. Validation is the demonstration of job-relatedness by showing the relationship between the selection procedure and job performance. If, in conducting the analyses of the contractor’s selection procedures, COs find that a selection procedure or criterion had an adverse impact, they must ask the contractor to provide all available information regarding its validation of the criterion in question. The contractor should provide information regarding the following:

• Development of the procedure and criterion, including, if the procedure is a written test, whether the contractor developed it internally or purchased it from a vendor;
• Validity studies it conducted, including the correlation coefficient (criterion and construct validation), or how the selection criterion or test content is linked with job requirements, that is, content validation;166
• Job analysis in the case of content validity studies and job information that describes how and why the selection criterion was chosen for criterion-related validity studies;167 and
• Consideration given to alternative selection criterion, the reason for rejecting the alternatives considered, and any efforts made by the contractor to lessen the adverse impact.168

COs must analyze this information, along with the analysis finding adverse impact caused by the use of the selection procedure and any other relevant evidence, in accordance with UGESP. Further analysis of the selection procedure or criterion at issue, or both, and their validation, will be coordinated between the regional office and the national office.169