Mum-of-three Ario Nepia uses social media platforms, like Facebook, to stay connected with whānau, but in recent weeks she learned photos of her and her family have been used without her consent alongside AI-generated explicit content.
Watch the full Marae report on TVNZ+
“I received a phone call from my sister. She sent me some screenshots on Messenger of a conversation that she had had with my niece who had informed her that she received a video, and that the contents of the video contained AI-doctored pornography in the likeness of some members of the community alongside mugshots.”
She said upwards of 15 members of the community featured in a gallery of photos.
As a mother you just go into panic, she said.
“I think your mind just starts running wild, you know? Why does this person, who I don't have a relationship with, have images of myself and my family, and photos from my wedding, on a gallery alongside AI-doctored pornographic material in the likeness of people.
“Why are all of these people's mugshots in this gallery? What does he intend to use the gallery for? How are these photos being distributed?”
She sought clarity and answers about how the material came to exist and how widely it may have been shared.
“My kids are my world. There are no relationships that to me are more important to protect than their safety.
“I will burn every relationship I have to the ground to protect my children, and I think that should also be the position of all parents. And from what I witnessed, my children are the youngest featured in that gallery, and I just can't, to this day, fathom how somebody - why somebody - would have those photos.”
A public apology has since been posted online acknowledging the harm caused. It states the person involved was a recipient of the material rather than its creator. Those claims have not been tested in court.
Nepia said she “absolutely” accepts the apology but still wants answers.
“I can accept that apology and still require the questions that are unanswered to be answered, and still ask more of the criminal justice system, and still ask more of our schools in terms of child protection.”
She said she is asking more from the community in response “because it is a community security violation - it hasn't happened to one individual, it has happened to multiple individuals.”
“How are we, as a community now, preparing for, number one, a world with artificial intelligence where it can be used in such a harmful way? How are we collectively learning from this experience in protecting our tamariki from being able to be exposed from this type of thing in the future?
“It's really opened up an entire world in terms of artificial intelligence and what it's capable of doing.”
AI-generated image abuse growing
New Zealand’s online safety organisation Netsafe said AI-generated image abuse is an emerging issue and encourages people to report harmful content as soon as they become aware of it.
In May, a 21-year-old was sentenced to 24 months' intensive supervision in the first prosecution of its kind in the country, for sharing pornographic images created by deepfake technology.
There is still a debate over whether New Zealand's laws are keeping pace with rapidly advancing AI technology.
ACT MP Laura McClure is sponsoring a bill that would criminalise creation and distribution of non-consensual, sexually explicit deepfakes and provide clearer pathways for harmful content to be removed.
Meanwhile, Nepia has gone from having an open Facebook page, sharing stories and content, to now locking her profile.
“[Facebook] can be used in a really beautiful way when it's not abused. So, yeah, I did have my profile open for [family] to be able to access photos and all of that, and I think, ‘Oh, I don't need to be doing that [locking public profile]’. I wish, now, I had done that from the very start. Like, I look back on that and I'm gutted with myself.”
Nepia hopes that what she and others have experienced encourages more parents to have conversations about online safety, artificial intelligence, and what young people are doing on their devices.
“We've all been guilty of being super busy and we see our kids on their phones scrolling, and don't bother to go over and grab the phone off them and do, even just a quick run through of what's happening on the phone and what are they doing on their phone.
“Have conversations with your kids. All I can say to people who are widely accepting – whether in private or public – accepting of this behaviour is I hope it never happens to you.
“I hope you're never on the receiving end. I hope you're not awake at night wondering and thinking where images of your children are and how they're being used. I hope that's never your reality because I can tell you now, it's not a reality I would wish upon anybody.”
SHARE ME