It might feel good to support a small Kiwi business, especially if there's a lovely local craftswoman behind it – but what if it’s nothing more than "AI slop"?
AI ads have been scamming craft-loving consumers into paying top dollar for cheap knock-offs and a convincing yarn.
An ad featuring 68-year-old Raewyn Thorburn, an Otago nana selling her final handmade baskets before retirement, turned out to be more than meets the eye – thanks to her curiously American lilt and unusual pronunciation of the region's name.
AI ads have been scamming craft-loving consumers into paying top dollar for cheap knock-offs and a convincing yarn. (Source: Seven Sharp)
That's because Raewyn didn't exist, and nor did her heart-tugging backstory.
Raewyn's also turned up in other places, hawking musical instruments as Maree – stock that can be found on Temu for a fifth of the price.
Kiwis caught in the scam have taken to social media to warn others.

So, when overseas AI fakes can get past our consumer laws, how do we know who to trust?
Dr Andrew Lensen, a senior lecturer in artificial intelligence at Victoria University, told Seven Sharp he watched the ad several times – and picked up new tell-tale signs with each rewatch.
One of the obvious tells, he said, was that the ad was made up of small clips joined together.
"We see that quite often in these sorts of formats because when you create a video, you can only make a short one – a few seconds," he said.

There were also inconsistencies in some of the background shots.
"You can see the background changes in different scenes; the captions don’t quite line up – those sorts of things. And it’s sort of oversaturated as well, so there’s lots of small signs but the voice, obviously, was the main one for sure."
Lensen said it’s "getting easier and easier" for people to create their own fictional salespeople – complete with a fake voice, backstory and workshop.
"A few years ago, you could do it, but you had to have a lot of skill, a lot of expertise, a lot of money or resources. Whereas now, there’s services online that we can almost sell this to you at quite a cheap price and so the barrier to entry’s a lot lower," he said.
People could churn out their own Raewyn after a few days of training, he said – and then make a workflow to repeat the whole process over and over again.

While shoppers have been warned to check for signs such as hands, Lensen said the clues were getting increasingly difficult to spot.
Accents would take "a little bit longer" for the AI to perfect, he said, adding that there was "not a huge amount of New Zealand voices out there in the training data sets".
"The amount the technology has improved over the last two or three years, you can just sort of see, again, in a year or two, it’s going to be even harder," he said.
Rather than keeping an eye out for minor discrepancies, Lensen urged people to independently verify sources.
"So, when you go and look at this product they’re trying to sell, can you find reviews of it? Can you do an image search and see where else it’s listed? In this case, it was also on Temu, I think, so those sort of critical thinking, independent checks are probably the way forward."
Lensen said it was "quite shocking" that the Raewyn ad had yet to be taken down despite major platforms such as YouTube having forms of detection in place to block them.
"I can still see it on there even know I know many, many people have reported it, so there’s a big question there as to why they’re not taking it down, even when they’re being told about it.
"I think we really should expect them to do more to stop this – both for their own reputation but also to protect their users."



















SHARE ME