Prejudiced AI systems could result in more Māori being arrested or falsely accused of crimes, expert warns

Source: 1News

Indigenous data specialist Karaitiana Taiuru says police’s new research programme into biases in the force is a positive, but he fears prejudiced AI systems will result in more Māori being arrested or falsely accused of crimes than non-Māori.

Taiuru said he has some reservations about the research programme announced last night — called Understanding Policing Delivery —which police say will focus on examining where bias may exist within policies, processes and practices.

The independent panel, chaired by longtime justice system critic Sir Kim Workman, will work with the University of Waikato’s Te Puna Haumaru NZ Institute for Security and Crime Science on the research.

“I think it‘s a positive step going forward. I’m glad Sir Kim is chairing that committee. I do have some reservations about academic restrictions on that research,” Taiuru said.

“Whether the research findings and terms of reference will be made public.”

Taiuru said he fears that racial profiling and photographing of Māori and Pasifika could escalate the issues currently in America, where facial recognition systems can’t differentiate between Black people.

“Those biases usually happen from the engineers who are normally middle class, white males who design a system based on what they consider to be normal — that’s themselves,” he said.

"My fear is that we will have the same issues in New Zealand. Will the system differentiate between one Māori male and another Māori male?

"Will it differentiate between a male with a full face tā mako and another male with a full face tā moko?

“I don’t believe the system has been trained to do that at this stage.”

Biases in the system could lead to even more Māori overrepresentation in the justice system, Taiuru said.

“Māori are overrepresented in the justice system, overrepresented with the police, so by default we will have more Māori photos going through the system,” he said.

“That system won’t recognise Māori with darker skin, won’t recognise the tā moko and differences.

“My concern is that we will see more Māori being arrested or being falsely accused than non-Māori, if the Māori are suspected of a crime of say more than two years, we might have more Māori forced to take a DNA sample for the police.

"The psychological issues, I can’t imagine being arrested because an AI system thought I was someone else.

“It creates mistrusts in communities. The issues go on and on."

Taiuru said a more diverse workforce reduces the risks of AI and facial recognition systems being programmed with biases, but that would take time.