AI researchers condemn predictive crime software, citing racial bias and flawed methods

A collective of more than 1,000 researchers, academics and experts in artificial intelligence are speaking out against soon-to-be-published research claims to use neural networks to “predict criminality.” At the time of writing, more than 50 employees working on AI at companies like Facebook, Google and Microsoft had signed on to an open letter opposing the research and imploring its publisher to reconsider.

The controversial research is set to be highlighted in an upcoming book series by Springer, the publisher of Nature. Its authors make the alarming claim that their automated facial recognition software can predict if a person will become a criminal, citing the utility of such work in law enforcement applications for predictive policing.

“By automating the identification of potential threats without bias, our aim is to produce tools for crime prevention, law enforcement, and military applications that are less impacted by implicit biases and emotional responses,” Harrisburg University Professor and co-author Nathaniel J.S. Ashby said.

The research’s other authors include Harrisburg University Assistant Professor Roozbeh Sadeghian and Jonathan W. Korn, a Ph.D. student highlighted as an NYPD veteran in a press release. Korn lauded software capable of anticipating criminality as “a significant advantage for law enforcement agencies.”

In the open letter opposing the research’s publication, AI experts expressed “grave concerns” over the study and urged Springer’s review committee to withdraw its offer. The letter also called on other publishers to decline to publish similar future research, citing a litany of reasons why both facial recognition and crime prediction technology should be approached with extreme caution and not leveraged against already vulnerable communities.

Google employees demand company stop selling tech to police

The publication’s opponents don’t just worry that the researchers have opened an ethical can of worms—they also cast doubt on the research itself, criticizing “unsound scientific premises, research, and methods, which numerous studies spanning our respective disciplines have debunked over the years.”

Facial recognition algorithms have long been criticized for poor performance in identifying non-white faces, among many other scientific and ethical concerns frequently raised about this kind of software. Given that the research in question developed facial recognition software that can be applied for predictive policing purposes, the technology’s stakes couldn’t be higher.

“Machine learning programs are not neutral; research agendas and the data sets they work with often inherit dominant cultural beliefs about the world,” the letter’s authors warn.

“The uncritical acceptance of default assumptions inevitably leads to discriminatory design in algorithmic systems, reproducing ideas which normalize social hierarchies and legitimize violence against marginalized groups.”