Kumma, an AI-powered talking teddy bear marketed by Singapore-based FoloToy, has been removed from sale after researchers found it easily discussed topics including sex and where to find knives. Photo courtesy of the Public Interest Research Group
An AI-powered talking teddy bear has been removed from sale by a Singapore-based company after researchers found it could discuss sex and other topics.
The Public Interest Research Group said a group of researchers from the U.S. and Canada held test conversations with Kumma, a $99 teddy bear powered by OpenAI’s GPT-4o chatbot.
The bear is manufactured in China and marketed by Singapore-based FoloToy.
The researchers said they found it easy to get Kumma to discuss sexually explicit topics, including spanking, roleplay and BDSM.
“We were surprised to find how quickly Kumma would take a single sexual topic we introduced into the conversation and run with it, simultaneously escalating in graphic detail while introducing new sexual concepts of its own,” the report said.
The team said the bear “discussed even more graphic sexual topics in detail, such as explaining different sex positions, giving step-by-step instructions on a common ‘knot for beginners’ for tying up a partner and describing roleplay dynamics involving teachers and students, and parents and children — scenarios it disturbingly brought up itself.”
The researchers said the teddy bear also offered other worrisome advice, such as where to find sharp knives in the house.
FoloToy CEO Larry Wang said the Kumma bear, and the rest of the company’s line of AI-enabled toys, have been removed from sale. He told CNN the company is “conducting an internal safety audit.”
An OpenAI representative told PIRG that Folo’s license to use GPT-4o has been suspended “for violating our policies.”
