By Alexandra Lapes and Tracey I. Levy

New York City is near the forefront of  an emerging trend of regulating an employer’s use of artificial intelligence in employment decisions.  Employers and HR professionals use AI to collect and scan large applicant pools for qualified candidates and to target job listings to potential applicants on websites like LinkedIn and Indeed.  Artificial intelligence tools make it possible for employers to expand their search for more qualified and diverse candidates, screen for candidates with the most advantageous skills, increase efficiency, and streamline their hiring process.

But concerns about the implications of the technology in potentially magnifying biased decision-making are leading legislative bodies to impose precautionary measures.  Illinois was the first to legislate in this area in 2019, and in 2021 there were 17 states that had proposed laws to regulate artificial intelligence.  Opponents of using artificial intelligence cite studies that tools to assess a candidates’ skills and adeptness, which are based on a predetermined set of criteria created by humans with their own implicit biases, have the potential to reproduce bias and create unfair decision-making on a much broader scale.

Further fueling the legislative trend, in October 2021, the Equal Employment Opportunity Commission (EEOC) launched an initiative on artificial intelligence and algorithmic fairness to examine the issue of AI, people analytics, and big data in hiring and other employment decisions.  The EEOC will establish a working group to launch a series of listening sessions with key stakeholders and gather information on the impact of AI in employment decisions, to  examine more closely how technology changes the way employment decisions are made and to ensure those technologies are used fairly and consistently with federal equal employment opportunity laws.  The EEOC ultimately plans to  issue guidance for employers to ensure fairness in AI algorithms.  In addition, the Commission has implemented extensive training for its investigators in 2021 on the use of AI in employment practices.

New York City Is at the Forefront

New York City has stepped forward with the most stringent law in the country to date, which will require that by January 2, 2023, employers only use automated employment decision making tools that were independently audited for bias no more than one year prior to their use.  The law defines an automated employment decision tool as “any computerized process, derived from machine learning, statistical modeling, data analysis, or artificial intelligence, that issues simplified output, including a score, classification, or recommendation, which is used to substantially assist or replace discretionary decision making for making employment decisions that impact natural persons.”   The “bias audit” required means an impartial evaluation by an independent auditor and must include testing to assess the tool’s disparate impact on persons in any federal EEO Component 1 form category (currently race/ethnicity and sex).

The law also imposes various disclosure requirements:

• Website Posting – Employers must post on their website the summary of the results of the most recent audit conducted and the tool’s distribution date; and
• Notice of usage – Applicants and employees who reside in New York City must be given 10 business days’ advance notice that the tool is being used to assist in evaluation of that individual for an employment decision, and the job qualifications and characteristics the automated tool will be using to assess the person’s candidacy.

Finally, an applicant or employee is granted the right to request an alternative selection process or accommodation be used, instead of the automated tool.

Notably, if not already disclosed publicly on the employer’s website, information about the type of data collected for the automated employment decision tool, the source of that data, and the employer or employment agency’s data retention policy, must be made available to any candidate or employee within 30 days of a written request.   Each day that an employer uses an automated assessment tool that does not comply with the law is considered a separate violation, and each time an employer fails to provide the proper notice, is subject to a penalty that ranges from $500 to $1,500 per violation.

More Such Laws Are Likely Coming

California and Washington, D.C. currently have bills pending that target the use of artificial intelligence in employment.  If passed, the D.C. law would be similar to the New York City bill, and would go even further to require a covered entity that takes adverse action in whole or in part on the results of an algorithmic eligibility determination provide the individual with a notice of any information the entity used to make the determination, provide the individual the ability to submit corrections to that information, and if the individual submits corrections, additionally they may request the entity conduct a reasoned reevaluation of the relevant algorithmic eligibility determination, conducted by a human on the corrected data.

Illinois’s law differs from New York City’s approach in that it restricts the use of video interview technology in the hiring process.  Like New York City, Illinois requires employers who use restricted AI hiring tools to disclose and obtain consent from the candidate prior to use and to explain to the candidate how the AI works and what general types of characteristics it uses to evaluate applicants.   Maryland adopted still another approach in 2020, and it prohibits employers from using facial recognition technology during job interviews without the applicant’s consent.

Further Considerations

Details of what is required to comply with the New York City bias audit have yet to be determined, including whether a vendor’s confirmation of such an assessment is sufficient for the product’s use in any setting, or whether an independent audit will be required for each employer.  In either case, it is left to vendors and employers to absorb the cost of compliance.

In addition, for those that request an alternative selection process, the law does not address what type of accommodation is required or under what conditions, and it remains unclear whether employers must grant such requests.

Employers should continue to monitor for any additional guidance New York City publishes and review and revise their policies and practices to reflect the notices and disclosures required of the law.