The (Not So) Hidden Bias in AI Hiring Tools

Nick: Welcome to the Wellable Weekly Podcast, where we talk about key topics and trends at the intersection of well-being, technology, and HR. I’m Nick, along with my good friend and colleague Geoff. 

All right, hopping into it — another AI article to kick us off. What’s good about this is that it’s not just an article, it’s actually a study, which I find really interesting. It’s from the Stanford University Human-Centered AI group, Stanford HAI. To set the foundation: a big part of our audience is involved in recruitment and retention, and an interesting stat from this study was that 90% of US employers use AI screening tools to sort and rank job seekers. That blew my mind. I know AI is really popular, and I always feel like as a company we’re fairly ahead of the times. We don’t do that. So I saw 90% and it was a curveball. I don’t understand how 90% of companies do anything — that’s a ton of alignment. 

What’s probably more interesting, going a layer deeper, is that the majority of those companies are working with the same few third-party models and vendors. So as an applicant, if I apply to 10 jobs, and I can now generate customized resumes and cover letters using AI — so I’m applying to more jobs than ever — and 5 of those 10 companies are using the same AI model, that model has now seen my resume five different times. Because it sees it multiple times, it’s using some of that prior data to render an opinion about me for each new position. Whereas in the real world without AI, you’d have five HR people at five different companies reviewing my resume each independently. 

The study covered 3.4 million people who submitted 4 million job applications, across 1,700 job postings, 150 employers, and 11 industries. It’s the first large-scale study of AI hiring, which makes it especially compelling. It found two major issues: a significant increase in racial bias, and a tendency to shut the same people out of jobs everywhere they apply. If you’re rejected from the first position, that rejection plays a factor in future applications to companies using the same model, and you can get effectively locked out even if you’re a strong fit. That’s bad for employers who miss good candidates and bad for employees who never get the chance to interview. 

Geoff: A few interesting stats worth unpacking. The applications across 1,700 job postings — that’s over 2,000 applications per job posting. That’s actually not that uncommon for larger companies that are appealing to new graduates looking for secure work with compelling benefits and pay. Because of that volume, and the ease with which people can apply now, it’s created a perfect storm of increased applications. In order to handle that volume, organizations are resorting to tools powered by a select few AI models. 

The term the article coins to describe this effect is algorithmic monoculture. That term actually comes from farming. The basic premise is that if every farm plants the same single crop strain, one disease could wipe everything out because there’s no diversity. The same logic applies here. When every employer uses the same AI vendor for screening, rejection decisions become correlated. Normally, applying to company A and company B should produce independent decisions. But if they’re running the same algorithm and that algorithm doesn’t like your profile, they can reject you for the exact same reason across the board. Most applicants aren’t even aware this could be happening when they submit their applications. 

Nick: Just one detail about the study worth noting: all the applications in this study were processed through the same AI hiring tool, so they could confirm every application was going through the same algorithmic model. That makes the findings cleaner. 

I always think about the concept of AGI — artificial general intelligence — which refers to AI that can be intelligent the way a human is. In an ideal world, an AI hiring tool would work like a recruiter who reviews each application independently. But a recruiter at company A is looking at a resume for the first time, and a recruiter at company B is also looking at it for the first time. Those evaluations are independent of each other. 

That independence disappears when it all feeds into a single model. These AI tools have four properties that fundamentally distinguish them from human recruiters: they’re pervasively adopted across many employers simultaneously, they have persistent memory across applications, their decisions are highly consequential, and they’re opaque — neither candidates nor employers can see exactly how decisions are being made. Even if AI were capable of true AGI-level reasoning, those properties create problems that don’t exist in the human recruiting process. 

Solving it is not straightforward. You could tell HR teams: if you see a resume that appeared in a prior screening, clear your history. But I don’t know if that works, because I don’t know if the problem is the memory or if the same model would just produce the same output regardless. 

Geoff: Right. The natural instinct for an HR leader reading this study is probably not to ditch the tool altogether but to look for configuration tweaks that preserve the efficiency gains. Given the volume of applications organizations are receiving every day, I don’t think many will abandon AI screening entirely. Some of the onus has to fall on the AI vendors to allow employer-level configuration and actually have those configurations affect how memory and screening are conducted. But I’m not sure how much will change quickly. 

Nick: It’s a tough place to be in HR. Even if your team stayed the same size, 3x the applications means you’re not equipped to handle that volume without AI. So there’s a real need for a solution. AI is the natural one. There are clearly real challenges with it, as this study shows. But just powering through triple the applications manually doesn’t seem like the right answer either. 

Which actually dovetails into our next story. Uber is planning to lay off 23% of its HR workforce. Before we get into the details — Uber has explicitly said this is not AI-driven, which is very different from almost every other layoff you’ve seen in the news lately. It’s hard to completely believe that, because Uber has been disproportionately in the news lately. First for its very strong stated intention to use AI to optimize its business. Then a report that they blew through their AI budget in the first three months of the year. Then the CTO came out and acknowledged they don’t have a very clear or demonstrable ROI from their AI spend. So before you hear about the layoff, what you know is that they’re spending aggressively on AI and don’t have much to show for it, which means they’re probably looking for cost savings. 

In this new environment, it feels like everyone is cutting middle managers. HR doesn’t have middle managers in the strict sense, but it has a middle-manager-adjacent dynamic of managing people and processes that makes it a common target. The CEO, in an internal memo, said the changes are necessary to maximize the effectiveness of the people team and the enormous potential ahead. That’s vague enough to mean almost anything. They later explicitly said this is not AI-driven, but I understand why that’s met with some skepticism. 

Geoff: Actions speak louder than words, in its most basic form. There’s so much scrutiny and skepticism around C-suite intentions and how layoffs are being communicated. We talked about this with the Bolt CEO, Ryan Breslow, making a big announcement about getting rid of his entire HR team — but he still had people ops covering those functions. You simply have to have these functions covered. Someone needs to own core people-related processes for the business to run smoothly. You can reframe it as HR versus people operations, that’s fine. But cutting the processes out entirely is not sustainable. It’s about how you structure it to make sure those things are still getting done. 

Nick: A similar analogy: I often get the question of whether a company should hire a dedicated wellness coordinator. And I come back to: what you really need is someone to own the wellness program. That could be a single person with wellness in their title, or it could be a wellness committee, or an HR benefits person who’s really engaged with wellbeing and wants to own the process. What you can’t do is launch wellness software without any true internal owner, because it will flop. Someone has to own it. The same logic applies to HR. Yes, it could be traditional HR. But the reality is all these processes need an owner — whether that’s HR, managers, or a combination. Uber is a sophisticated company. They’re not going to forget payroll or benefits. They’re just trying to distribute those responsibilities differently, probably eliminating the middle layer and pushing ownership directly to managers and the rank and file. 

Geoff: Yeah, makes sense. Well, that seems like a good place to wrap up. Thanks as always for tuning in. You can catch us on Spotify, Apple Podcasts, or wherever you get your podcasts. And be sure to subscribe to Wellable Weekly for all of our latest insights. Thank you.