Should an employer have to tell you when it uses a hiring algorithm?

A New York City councilwoman wants to take the opaqueness out of algorithms used in job recruitment. Laurie Cumbo's proposal, presented to the New York City Council, would require employers and managers to be absolutely transparent about their use of software when it comes to hiring employees. Further, she wants the makers of these algorithms to commit to regularly updating them to ensure they don't discriminate — a sensible request given how often we've seen software mirroring the prejudices of its makers.

Algorithms are increasingly used to whittle down applicant pools, but their use is rarely (if ever) disclosed to applicants. If successful, Cumbo's legislation will take effect in January 2022. And she's not alone in pushing such legislation. At least 10 senators have similarly implored the Equal Employment Opportunity Commission to seriously assess the bias in artificial intelligence-based hiring processes.

Algorithms determine your employability — Automated employment-decision technology runs the show for many employers right now. At least 67 percent of hiring managers on LinkedIn have said that they use algorithms to find the right applicant. These algorithms act like virtual recruiters that screen resumes and pick out the "perfect" candidate for an open role based on internal requisites of the companies that use them.

But "perfect" here, critics worry, is a criterion that is rarely clearly defined and algorithms have the potential to magnify confusion and bias against an applicant based on their race, sex, religion, and other background metrics. On top of that, as the New York City proposal warns, such software is rarely vetted, nor is its use disclosed to the public, which makes it all the more difficult to ensure that it isn't disadvantaging some candidates compared to others.

These worries aren't unfounded — Amazon, the retail behemoth that made Jeff Bezos the world's wealthiest person (though Elon Musk is nipping at his heels), had to do away with a recruitment algorithm because it discriminated against female applicants in 2018.

For years, mathematicians like Cathy O'Neil have warned against placing blind faith in algorithms used to hire people precisely because of the inevitable biases within them. Even systems trained on previous hiring data are doomed to replicate those decisions, so unless previous hiring was neutral, they'll simply replicate the prejudices people hope to avoid. Beyond the scope of hiring, algorithms are well documented to exacerbate bias, from facial recognition tools to harming Black patients' odds of receiving kidney transplants.

Right now, Cumbo's proposal faces some resistance from politicians who want to modify the legislation's requisites. Some worry it will hurt employers' ability to find effective candidates, but others have supported Cumbo and noted that the proposal could improve people's access to jobs. It's destined to be an ongoing and heated debate.