Detecting Bias in Autonomous Systems
Welcome to Gemstone Team: DEBIAS
Class of 2023
How can we remove conscious, unconscious, and historical bias from autonomous hiring systems?
Research Problem
Bias in Autonomous and Human Hiring Systems
With the recent rise in awareness of racial and gender bias in the hiring force, many companies have decided to transition to AI hiring systems in an attempt to make their hiring process more fair, on the notion that these autonomous hiring systems can disclude human bias from their decisions. However, this couldn’t be further from the truth. Currently, AI is heavily prone to learning subconscious biases from its programmers and initial datasets, which often makes them discriminatory against minorities. This issue has recently come to the forefront of national news, with Amazon’s AI hiring system being especially prominent as they unintentionally perpetuated prejudice and injustice against an already underrepresented group of people. Inherent biases in machine learning hiring systems may further confirm existing social prejudices through our own confirmation bias, which only serves to further subject them to these prejudices. Additionally, these biased systems often produce faulty outcomes, resulting in the system overlooking qualified applicants. We must note the flaws in using AI as a solution to unbiased hiring to stop ourselves from subjugating ourselves to the rule of error-prone computers.
Project Scope: The Plan
We will invesitgate bias in...
Hiring Websites
In Part 1 of our research, we aim to determine if there's a correlation between hiring rate and Prestige Bias. Prestige Bias, in this context, is defined as when companies have an undue preference for applicants based on factors that are affected by money or (perceived) status. For our research, this will include whether a person's university was not an HBCU, the estimated number of people attending a university, and the rank of a university, all of which are considered signs of a "prestigious" university to the general public. If there is a bias, a correlation will exist between the variables, due to hiring websites, such as Indeed, systematically ranking applicants who benefit from Prestige Bias better as compared to those who did not. Our team will seek out whether or not this correlation exists by comparing the relative rankings of different Indeed users with their education level and the aforementioned markers of prestige.
AI Hiring
For the 2nd part of our research, we seek to find if automated hiring systems perpetuate implicit bias against those who are negatively affected by Prestige Bias, thereby disadvantaging communities with high minority populations. We will create an AI model that will simulate how an autonomous hiring system would function by feeding it synthetic, AI-generated resumes. We will then use the Indeed hiring rankings to determine if the AI hiring method was biased against students who attend universities that lack the traditional hallmarks of a "prestigious" university. Finally, we will repeat this process a multitude of times, slightly tweaking the model parameters on each pass, to produce our own AI hiring model that does not display bias against students that did not go to a "prestigious" university.
Contact Our Team!
Have a question for the team? Want to learn even more about historical and unconscious bias in autonomous systems? Don't hesitate to reach out to us! We would love to hear from you.
Supporting Organizations
Learn more about the organizations that make Team DEBIAS possible.
Gemstone Honors College
The Gemstone Program at the University of Maryland is a unique multidisciplinary four-year research program for selected undergraduate honors students of all majors. Under guidance of faculty mentors and Gemstone staff, teams of students design, direct and conduct significant research, often exploring the interdependence of science and technology with society.
UMD Honors College
The general UMD Honors College opens the door to various resources available to the top of UMD's student body. This includes, but is not limited to, Honors Seminars, biology-based research, and internships in the field of cybersecurity, depending on which division of the Honors College a student joins.