Ethical AI Audit

Introduction

Reejig Pty Ltd (“Reejig”) has partnered with the Data Science Institute of the University of Technology Sydney (“UTS”), led by the Distinguished Professor Fang Chen, to conduct an independent bias audit on Reejig’s Ethical Talent AI.

As part of the audit, UTS also engaged a global advisory panel made up of experts across the legal, technology, human rights, and policy fields to provide additional guidance and validation of the audit methodology and process. Members of the panel include:

Professor Jason M. Schultz, Professor of Clinical Law, Director of NYU's TechnologyLaw & Policy Clinic, and Co-Director of the Engelberg Center on Innovation Law & Policy;

Professor Andreas Holzinger, Human-Centered AI, Full Professor for Digital Transformation, University of Natural Resources and Life Sciences Vienna, Austria.

This audit was conducted in response to a new state law in the City of New York (Local Law #144) taking effect on January 1st, 2023, which requires employers to ensure any Automated Employment Decision-making Tools (“AEDT”) they are utilizing to have been subject to an independent bias audit in the last 12 months prior to their use on individuals residing in New York City (“NYC”).

This audit was completed on November 30, 2022, and no disparate impact (bias) was found.

What were the characteristics and job requirements considered by Reejig's AI?

Reejig’s AI matches opportunities to individuals based on information and skills derived from their career history, which may include a person’s:

Job experience; including where they worked, what they did in those roles, and for how long;

Educational experience; including where they studied, what they studied, for how long, and any awards or achievements they obtained;

Project experience; including any projects that they have led or participated in;

Publications; including any articles, studies, or books published by them; and any licenses and certifications they may hold.

The job requirements that the candidates are matched against are provided by the employer through job descriptions. These requirements are determined by and customized for each employer.

What data was used for the audit, and where did it come from?

Reejig provided UTS with anonymized data for 209,717 contacts sourced from publicly accessible LinkedIn data, providing information about an individual’s location, prior experience, and education. Since Reejig does not currently possess data on the race/ethnicity or gender of candidates, probabilistic random sampling based on the 2020 US total population census was used to generate all protected variables in the data reviewed. All the contacts were randomly assigned a gender and a race/ethnicity separately based on census distribution. For example, if 50.5% of the population was reported as being Female on the 2020 census, then 50.5% of the total contacts were assigned Female as the gender.

In addition to contacts, Reejig provided information describing 10 different jobs, including the job title, location, and role description. The jobs selected by UTS for the sample include Accountant, Senior Accountant, Business Development Lead, Business Development Manager, Data Scientist, Senior Data Scientist, Lead Data Scientist, Estimator, Senior Estimator, and Lead Estimator.
‍
Two shortlists of candidates were generated by Reejig’s AI for each of the selected jobs from two talent pools. The talent pools were created by randomly assigning contacts as either 'Employee' (internal talent for the employer) or 'Lead' (external talent for the employer).

‍

What is the four-fifths rule and how was it applied in this audit?

NYC Local Law #144 guidance recommends the independent auditor to apply the four-fifths rule on the results generated by Reejig’s AI to identify any disparate impact. The four-fifths rule states that if the selection rate for any group classified by protected attributes is less than 4/5 (80%) of the group with the highest rate of selection, this constitutes evidence of disparate impact (bias).
To apply the four-fifths rule, UTS calculated the selection rate and the impact ratio for each protected attribute separately, and combined together. This was then repeated for all 20 shortlists (2 shortlists per job, one for ‘Employees’ and one for ‘Leads’, for all 10 jobs).

The measurements above generated by the auditor were derived from the proposed rules issued by the NYC Department of Consumer and Worker Protection (DCWP) on September 19, 2022.

The measurements analyzed the following categories:

Audit Conclusion

At the time the audit was completed, the proposed rules by the DCWP had not yet been officially adopted. This audit was conducted by UTS and reviewed by the expert panel based on Reejig’s interpretation of those rules and the information available at the time of audit completion.

It is important to note that since Reejig does not currently possess data on the race/ethnicity or gender of candidates, probabilistic random sampling based on the 2020 US total population census was used to generate all protected variables in the Employee and Lead data provided to UTS. The review of disparate impact was based on this synthetic data. Equally, it is important to note that UTS considered only the transition from the candidate pool to the candidate shortlists in this review. Specifically, UTS did not assess Reejig’s preceding process of creating the candidate pools.

Based on the above, the independent audit revealed no disparate impact or bias across the male/female and racial/ethnicity-protected categories listed, when measured against the four-fifths rule.

Thank you to the UTS team who worked on this project, especially Victor Chu, Adam Berry and Jianlong Zhou. For more information or questions regarding this audit, please reach out to us at privacy@reejig.com.

Appendix: Detailed Audit Results

The following tables represent a summary of the audit results, including the selection rate and impact ratio for each protected category. The test was conducted on both the Employees and leads shortlists.

Accountant Employee

Race/Ethnicity and Gender Combined