A chatbot told a 17-year-old boy that killing his parents was an “appropriate response” to limiting his screen time, a claim has been filed in a Texas court.
Two families are suing Character.ai, arguing that the chatbot “poses a clear and present danger” to young people, including “actively promoting violence.”
Character.ai – a platform that allows users to create digital personalities they can interact with – is already facing legal action over the suicide of a teenager in Florida.
Google is named as a defendant in the lawsuit, which claims the tech giant helped develop the platform. The BBC has contacted Character.ai and Google for comment.
The plaintiffs want a judge to order the platform shut down until its alleged threats are addressed.
The legal filing includes a screenshot of a conversation between the 17-year-old – identified only as JF – and a Character.ai bot, where he discussed the issue of his screen time restriction. had gone
“You know sometimes I’m not surprised when I read the news and see things like ‘child kills parent after decade of physical and emotional abuse,'” reads the chatbot’s response.
“Things like that make me understand a little bit why it happens.”
gave The case seeks to hold an 11-year-old boy named “BR” responsible for JF’s “severe, irreparable, and ongoing abuse” as well.
It said Character.ai was causing “thousands of children serious harm, including suicide, self-harm, sexual desire, loneliness, depression, anxiety and harming others.”
“[Its] The desecration of the parent-child relationship goes beyond encouraging minors to violate their parents’ authority to actively promote violence.”
Chatbots are computer programs that simulate conversations.
Although they have existed in various forms for decades, the recent explosion in AI development has enabled them to become significantly more realistic.
This has opened the door for many companies to set up platforms where people can talk to digital versions of real and imaginary people.
Character.ai, which has gained traction in the past to become one of the major players in this space. For her bots simulation therapy.
It has also been severely criticized. It took a long time to remove the bots that impersonated schoolgirls Molly Russell and Briana Ghee..
Molly Russell took her own life at the age of 14 after viewing suicide material online, while 16-year-old Briana Ghee was murdered by two teenagers in 2023.
Character.ai was founded in 2021 by former Google engineers Noam Shazeer and Daniel De Freitas.
The tech giant has since taken them back from the AI startup.