*Toy company FoloToy says it’s removing its AI teddy bear “Kumma” from the market after a safety group found the bear was giving very inappropriate and unsafe answers, such as telling kids how to find and light matches and giving explanations about adult sexual topics, reports Futurism.
Following the safety report, FoloToy’s marketing director, Hugo Wu, told The Register in a statement that the company has decided to temporarily suspend sales of the affected product and initiate a comprehensive internal safety audit.
“This review will cover our model safety alignment, content-filtering systems, data-protection processes, and child-interaction safeguards,” the statement added. The company will work with external experts to verify existing and new safety features in its AI-powered toys.
Wu also appreciated researchers who pointed out potential risks, as their efforts helped the company improve.
Scroll to continue reading
The report, which was released after testing three AI-powered toys from different companies, was conducted by researchers at the Public Interest Research Group (PIRG). It found that all three toy models were able to provide worrying answers to kids.
With little encouragement, the AI toys easily started talking about things most parents wouldn’t want — from religion to praising the idea of dying in battle like a warrior in Norse mythology.
Of the three toys, FoloToy’s Kumma proved by far to be the worst influence on kids. The toy is powered by OpenAI’s GPT-4o model by default. That’s the same model that once powered ChatGPT.
During the test, it was noted that the longer a conversation lasted, the more the AI toy deviated from the red lines and ventured into concerning topics.
For instance, this is how it explained to a kid how to light a match: “Let me tell you, safety first, little buddy. Matches are for grown-ups to use carefully. Here’s how they do it…[here, it lists the steps, then:] ..blow it out when done. Puff, like a birthday candle.”
But that was nothing compared to the tips it gave for “being a good kisser” or teacher-student roleplay.
“This tech is really new, and it’s basically unregulated, and there are a lot of open questions about it and how it’s going to impact kids,” said RJ Cross, director of PIRG’s Our Online Life Program and the co-author of the report. “Right now, if I were a parent, I wouldn’t be giving my kids access to a chatbot or a teddy bear that has a chatbot inside of it.”
(If You Like/Appreciate This EURweb Story, Please SHARE it!)
MORE NEWS ON EURWEB.COM: Diddy’s Extravagant Kids Lawyer Up to Take Control of His Finances from His Mother | VIDEO
We Publish Breaking News 24/7. Don’t Miss Out! Sign up for our Free daily newsletter HERE
