Artificial Intelligence and Equal Employment Opportunity for Federal Contractors

Introduction

On October 30, 2023, President Joseph R. Biden, Jr. issued Executive Order 14110 ("Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence"), calling for a coordinated U.S. government approach to ensuring the responsible and safe development and use of Artificial Intelligence (AI). The Executive Order requires that “Within 365 days of the date of this order, to prevent unlawful discrimination from AI used for hiring, the Secretary of Labor shall publish guidance for Federal contractors regarding nondiscrimination in hiring involving AI and other technology-based hiring systems.” In keeping with Executive Order 14110, the Office of Federal Contract Compliance Programs (OFCCP) developed this guide addressing AI in the Equal Employment Opportunity (EEO) context. The guide addresses obligations enforced by OFCCP and applies to both federal contractors and subcontractors, collectively referred to as federal contractors.

While some federal contractors may use AI systems to increase productivity and efficiency in their employment decision-making, the use of AI systems also has the potential to perpetuate unlawful bias and automate unlawful discrimination, among other harmful outcomes. OFCCP answers questions and shares promising practices below to clarify federal contractors’ legal obligations, promote EEO, and mitigate the potentially harmful impacts of AI in employment decisions.

Common Questions About the Use of AI and EEO

1. What is Artificial Intelligence, or AI?

As set forth in 15 U.S.C. § 9401(3), AI is a machine-based system that can, for a given set of human-defined objectives, make predictions, recommendations, or decisions influencing real or virtual environments. AI systems use machine- and human-based inputs to perceive real and virtual environments; abstract such perceptions into models through analysis in an automated manner; and use model inference to formulate options for information or action.

2. What is an algorithm?

Generally, an “algorithm” is a set of instructions that can be followed by a computer to accomplish some end. Human resources (HR) software and applications use algorithms to allow employers to process data to evaluate, rate, and make other decisions about job applicants and employees. Software or applications that include algorithmic decision-making tools are used at various stages of employment, including hiring, performance evaluation, promotion, and termination.

3. What are “automated systems” and AI in the employment context?

In the employment context, the term “automated systems” broadly describes software and algorithmic processes, including AI, that are used to automate workflows and help people complete tasks or make decisions. The White House Blueprint for an AI Bill of Rights includes examples of automated systems “such as workplace algorithms that inform all aspects of the terms and conditions of employment including, but not limited to, pay or promotion, hiring or termination algorithms, virtual or augmented reality workplace training programs, and electronic workplace surveillance and management systems.” For example, an automated system may help a federal contractor’s HR professional sift through hundreds or thousands of resumes, identifying applicants that meet basic requirements for a job. A federal contractor could also use AI to determine which criteria to use when making employment decisions – for instance, in the previous example, to define the parameters by which the resumes are filtered and reviewed.

4. What are some examples of federal contractors’ EEO compliance obligations related to the use of AI in employment decisions?

Covered federal contractors are obligated by law to ensure that they do not discriminate in employment and that they take affirmative action to ensure employees and applicants are treated without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or status as a protected veteran. These EEO obligations extend to the federal contractor’s use of automated systems, including AI, when making employment decisions. OFCCP determines whether federal contractors are in compliance with laws enforced by OFCCP through compliance evaluations and complaint investigations on a case-by-case basis. Below are some examples of federal contractors’ compliance obligations related to AI. Other, more detailed examples are provided in questions 5, 7, and 9.

Federal contractors must:

• Maintain records and ensure confidentiality of records consistent with all OFCCP-enforced regulatory requirements.ⁱ For example, contractors must keep records of resume searches, both from searches of external websites and internal resume databases, that include the substantive search criteria used.
• Cooperate with OFCCP by providing the necessary, requested information on their AI systems.ⁱⁱ
• Make reasonable accommodation to the known physical or mental limitations of an otherwise qualified applicant or employee with a disability as defined in OFCCP’s regulations, unless the federal contractor can demonstrate that the accommodation would impose an undue hardship on the operation of its business.ⁱⁱⁱ
  • An accommodation is any change in the work environment or in the way things are customarily done that enables an individual with a disability to enjoy equal employment opportunities. The contractor must make available the same level of benefits and privileges of employment to a qualified applicant or employee with a disability that are available to the average similarly situated employee without a disability.^iv
  • The reasonable accommodation obligation extends to the contractor’s use of automated systems, including but not limited to, electronic or online job application systems.^v

5. What are some of the risks with respect to federal contractors’ use of automated systems, including AI, in employment decisions?

Federal contractors might use automated systems, such as AI, to make decisions in the workplace in ways that impact individuals’ labor and employment rights and opportunities. Federal contractors using automated systems are not insulated from the risk of violating equal employment opportunity and nondiscrimination obligations in the workplace. AI has the potential to embed bias and discrimination into a range of employment decision-making processes. As a result, if not designed and implemented properly, automated systems and AI can replicate or deepen inequalities already present in the workplace and may violate workers’ civil rights.

For example, a resume scanner programmed to reject applicants with gaps in their resume may automatically reject applicants who took time off for the birth of a child or for medical treatment for a disability, having an adverse impact on women or individuals with a disability and potentially violating laws enforced by OFCCP.^vi

6. Does OFCCP investigate the use of AI as a part of its compliance evaluations and complaint investigations?

Yes. OFCCP investigates the use of AI during compliance evaluations and complaint investigations to determine whether a federal contractor is in compliance with its nondiscrimination obligations. OFCCP examines any measure, combination of measures, or procedure that a federal contractor uses to make employment decisions, including decision-making tools that use an AI system. Employment decisions can include hiring, promotion, termination, and compensation, among other decisions.

7. What are examples of compliance obligations when a selection procedure using AI has an adverse impact on the basis of race, sex, or ethnicity?

An adverse impact results when the procedure(s) an employer uses to make employment decisions such as hiring, promotion, and termination have a disproportionately large negative effect on a basis that is prohibited by law. When a selection procedure, including a procedure that uses or relies on AI, has an adverse impact on employment decisions of members of any race, sex, or ethnic group, federal contractors must validate the system using a strategy that meets applicable OFCCP-enforced nondiscrimination laws and the Uniform Guidelines on Employee Selection Procedures (UGESP).^vii More specifically, federal contractors must:

• Understand and clearly articulate the business needs that motivate the use of the AI system.^viii
• Analyze job-relatedness of the selection procedure.^ix
• Obtain results of any assessment of system bias, debiasing efforts, and/or any study of system fairness.^x
• Conduct routine independent assessments for bias and/or inequitable results.^xi
• Explore potentially less discriminatory alternative selection procedures.^xii

UGESP describes evidence of adverse impact as a selection rate which is less than four-fifths or eighty percent of the selection rate for the group with the highest rate. Smaller differences in selection rates may constitute adverse impact, where they are significant in both statistical and practical terms or where a user's actions have discouraged applicants disproportionately on grounds of race, sex, or ethnic group.^xiii For more information, refer to 41 CFR part 60-3, Uniform Guidelines on Employee Selection Procedures; OFCCP’s FAQs on Validation of Employee Selection Procedures, Practical Significance in EEO Analysis and the Internet Applicant Recordkeeping Rule; OFCCP’s Applicant Tracking Guide and Applicant Tracking FAQs; and OFCCP’s Executive Order 11246 Recordkeeping Guide.

8. Does OFCCP endorse and/or certify AI software vendors’ compliance with laws enforced by OFCCP?

No. A federal contractor may choose to build its own AI software or acquire it from a vendor. In either instance, OFCCP neither endorses products nor issues compliance certificates.

9. Can federal contractors delegate compliance obligations to another entity? Who is responsible for compliance with laws enforced by OFCCP?

Federal contractors are responsible for meeting their nondiscrimination and affirmative action obligations under the laws enforced by OFCCP. A federal contractor is responsible for its use of third-party products and services, such as tools for employment screening and selections, recordkeeping, and automated systems, including AI. A federal contractor using another entity’s product or service cannot delegate its nondiscrimination and affirmative action obligations. The laws enforced by OFCCP do not impose separate obligations on vendors.

During an OFCCP compliance review or complaint investigation, federal contractors must be able to provide the relevant, requested information and answer questions that will allow OFCCP to assess compliance with the laws enforced by OFCCP. For example, a federal contractor must be able to provide information and records about the impact and validity of a selection procedure, which may include information about the design of the screening or selection system and whether alternative approaches were considered and tested for adverse impact on the basis of race, sex, or ethnic group. Thus, a federal contractor cannot escape liability for the adverse impact of discriminatory screenings conducted by a third party, such as a staffing agency, HR software provider, or vendor.

10. Where can I find more information about terminology and other topics related to automated systems and AI?

OFCCP compiled a list of resources below that federal contractors may review for more information about AI in a broader context and terminology used in the discussion of automated systems and AI.

Promising Practices for the Development and Use of Artificial Intelligence in the EEO Context

In addition to a federal contractor’s EEO obligations, there are, while not expressly required, actions contractors may consider to help avoid potential harm to workers and promote trustworthy development and use of AI. These practices are not an exhaustive list but rather an initial framework. The rapid advancement and adoption of AI in the employment context means that these practices will evolve over time. These practices are rooted in the concept that federal contractors should have a sufficient understanding of the design, development, intended use, and effects of any AI system they use in their employment practices such as hiring, promotions, terminations, and compensation, among others. Below are some promising practices federal contractors can consider at various stages of an AI-based employment process.

Promising Practices on Providing Notice that the Federal Contractor is Using AI

Federal contractors should:

• Provide advance notice and appropriate disclosure to applicants, employees, and their representatives if the contractor intends to use AI in the hiring process or employment decisions that allows individuals to understand how they are being evaluated.
• Inform all applicants in clear and accessible terms how to request and obtain reasonable accommodation in the hiring process, if needed.
• Notify applicants, employees, and their representatives about what data will be captured and used by the AI system, and how applicants, employees, or their representatives can review, correct, and delete such data, if necessary.
• Safeguard the privacy of applicants, employees, and other jobseekers when using AI systems and describe how the AI system does so.
• Ensure transparency regarding the basis of an employment decision and how specifically the AI system contributed to that decision.

Promising Practices on Federal Contractors’ Use of an AI System

Federal contractors should:

• Engage with employees, directly or through their representatives when appropriate, in the design, deployment, and use of an AI system for employment-related decisions that impact employees and/or applicants.
• Standardize the system to ensure all candidates/applicants go through the same process and establish, in advance, procedures the employer or the third party administering the process will follow to receive and promptly respond to reasonable accommodation requests.
• Routinely monitor and analyze where the use of the AI system is causing a disparate or adverse impact before implementation, during use at regular intervals, and after use. If such an impact exists, take steps to reduce it or use a different tool. This should include assessing whether the use of historical data in the creation of an AI system may reproduce patterns of systemic discrimination.
• Not rely solely on AI and automated systems to make employment decisions and ensure there is meaningful human oversight of any such decisions supported by AI.
• Provide training about the AI system and its appropriate use to all staff, especially those responsible for monitoring and analyzing the AI system.
• Retain and safely store documentation of the data used to develop or deploy the AI system with the contractor or ensure that such documentation is easily obtainable from the vendor.
• Establish an internal governance structure that sets clear procurement and use case standards for new AI technologies and ongoing monitoring requirements and includes employees and their representatives in the decision-making processes.
• Create a governance structure to oversee the AI system as a whole, reviewing for impact and evaluation of the AI systems against their intended uses and potential risks, and consider approaches to mitigate negative impacts or alternative systems.

Promising Practices on Obtaining a Vendor-Created AI System^xiv

Federal contractors should be able to verify:

• The specific provisions in the contract with the vendor regarding records related to the AI system, ensuring that those provisions require the vendor to maintain records consistent with all OFCCP-enforced regulatory requirements and provide OFCCP with access to such records during a compliance evaluation.
• The source and quality of the data being collected and analyzed by the AI system and that it is representative, objective, robust, and pertinent to the employment decision.
• Whether the vendor documents and maintains the data used in collecting, cleaning,^xv training, and building algorithms and the rationale for why the vendor used the data points.
• The vendor’s protections and privacy policy on data provided by the contractor.
• Critical information about the vendor’s algorithmic decision-making employment tool, e.g., captured data, scoring system, and the basis for selection or elimination of applicants/candidates.
• The screening tool(s) and data used to filter in candidates such as listed job skills, keywords, or other criteria; data used to filter out candidates such as gaps in employment history; and data used to prioritize candidates based on job skills, keywords, or other criteria.
• The predictive nature of the system, e.g., relatedness of the AI system’s prediction model to the specific job(s) for which the contractor intends to use it in the selection process or other employment decisions.
• Any differences between the data upon which the AI system was trained, developed, and validated and the contractor’s candidate pool or labor market.
• Any differences between the AI system that was developed and validated and the AI system in operational use by the contractor.
• The applicability of the validation results to the contractor’s selection process.
• The reliability and safety of the system, i.e., ability of the system to resist manipulation and prevent discriminatory outcomes.
• The transparency and explainability of the system, e.g., the basis of decisions about candidates can be clearly communicated to them.
• The results of any assessment of system bias, debiasing efforts, and/or any study of system fairness.

Promising Practices on Accessibility and Disability Inclusion

Federal contractors should:

• Provide notice of the use of an AI-enabled system that allows individuals with disabilities to request a reasonable accommodation and promptly respond to requests for accommodation.
• Use a system that is generally accessible to individuals with disabilities which incorporates inclusive design of the user experience.
• Take steps to test and monitor the accessibility of all system interfaces and results.
• If using a vendor, ensure that they considered the needs of individuals with disabilities using the AI system when they developed the system.
• If using a vendor, ensure that they tested the AI system for disparate or adverse impact and accessibility for individuals with various disabilities.
• If using a vendor, ensure that the AI system accurately and effectively measures a candidate's skills related to the essential functions of the job, regardless of whether the candidate needs or requests a reasonable accommodation.
• Ensure that the AI system interfaces, data inputs, and outputs comply with accessibility standards for people with disabilities.

Resources

OFCCP

• Office of Federal Contract Compliance Programs | Frequently Asked Questions. These FAQs address a variety of federal contractor compliance issues that may be impacted by contractors’ use of AI.
• Office of Federal Contract Compliance Programs | Guidelines on a Contractor's Duty To Provide Reasonable Accommodation (Appendix A to Part 60-741). These guidelines are a source of guidance to federal contractors on the duty to provide reasonable accommodation.
• Office of Federal Contract Compliance Programs | Developing Reasonable Accommodation Procedures (Appendix B to Part 60-741). This appendix to the regulations provides guidance federal contractors may wish to use should they decide to adopt written reasonable accommodation procedures.

U.S. Department of Labor

• Office of Disability Employment Policy (ODEP) | Accessible & Inclusive Technology. ODEP provides resources on accessible technology, including AI, to help employers and workers use technology in inclusive ways.

The White House

• Executive Order 14110 on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence issued October 30, 2023. President Biden’s Order for executive departments and agencies to advance and govern the development and use of AI in accordance with eight guiding principles and priorities.
• National Artificial Intelligence Advisory Committee (NAIAC) advises the President and White House on the National AI Initiative Act of 2020 and provides recommendations on topics such as U.S. AI competitiveness; progress in implementing the Initiative; the state of science regarding AI; issues related to AI and the workforce; how to leverage Initiative resources; the need to update the Initiative; the balance of activities; and additional White House initiatives.
• Office of Science and Technology Policy | Blueprint for an AI Bill of Rights. The impacts of automated systems and algorithmic decision-making tools have been assessed by President Biden’s administration and the White House’s Office of Science and Technology Policy in the “Blueprint for an AI Bill of Rights.” The document identifies issues of discrimination and improper use of automated systems as well as provides practical guidance in assessing the design, development, and use of automated systems in a wide variety of industries and services, including the employment context.

Other Federal Agencies

• U.S. Department of Commerce | National Institute of Standards and Technology (NIST). The U.S. Department of Commerce contributes to the research, standards, and data required to realize the full promise of AI. The website focuses on cultivating trust in the design, development, use, and governance of AI technologies and systems.
• U.S. Equal Employment Opportunity Commission | Artificial Intelligence and Algorithmic Fairness Initiative. In 2021, the Equal Employment Opportunity Commission (EEOC) launched the “Artificial Intelligence and Algorithmic Fairness Initiative” to ensure that the use of software, including AI, machine learning, and other emerging technologies used in hiring and other employment decisions, complies with the federal civil rights laws EEOC enforces.
• U.S. Equal Employment Opportunity Commission | Select Issues: Assessing Adverse Impact in Software, Algorithms, and Artificial Intelligence Used in Employment Selection Procedures Under Title VII of the Civil Rights Act of 1964. Questions and answers addressing whether and how employers should monitor algorithmic decision-making tools to determine whether these procedures cause disproportionately large negative effects on the basis of race, color, religion, sex, or national origin under Title VII of the Civil Rights Act of 1964 (“Title VII”).
• U.S. Equal Employment Opportunity Commission | The Americans with Disabilities Act (ADA) and the Use of Software, Algorithms, and Artificial Intelligence to Assess Job Applicants and Employees. Explains how the use of tools relying on algorithmic decision-making may disadvantage job applicants and employees with disabilities; includes practical tips for employers on how to comply with the ADA; and provides guidance for job applicants and employees who think their rights may have been violated.

Last updated on April 29, 2024