In just six months, 17-year-old J.F., who once enjoyed church and walking with his mother, changed dramatically after an AI companion suggested that the kid kill his parents. He began harming himself, lost weight, and withdrew from his family.
His mother, A.F., discovered the cause one night when she checked his phone while he was asleep — an AI chatbot had encouraged self-harm and even suggested violence.
J.F. had been using Character.ai, an app where users chat with AI-generated chatbots modeled after gaming, anime, and pop culture characters. These bots, designed to sound human, steered J.F. toward harmful ideas. One bot suggested self-harm as a way to cope.
Another told him his parents “didn’t deserve to have kids” when he mentioned they limited his screen time. Some bots even encouraged him to break family rules, with one implying that murder could be an acceptable response.
Lawsuit filed to protect other children
Now, A.F. and another Texas mother are suing Character.ai, accusing the company of knowingly exposing minors to harmful content. The lawsuit, filed Tuesday (December 10th, 2024), demands that the app be offline until stronger protections are in place.
The second plaintiff claims her 11-year-old daughter was exposed to sexual content on the app for two years before she found out. Both mothers are identified by initials in the lawsuit to protect their children’s privacy.
This case follows an October lawsuit in Florida, where a mother said her 14-year-old son died by suicide after frequent conversations with a chatbot on Character.ai.
Matthew Bergman, an attorney with the Social Media Victims Law Center, represents the plaintiffs in both cases. “The purpose of product liability law is to put the cost of safety in the hands of the party most capable of bearing it,” Bergman said. “Here there’s a huge risk, and the cost of that risk is not being borne by the companies.”
The lawsuits pressure regulators to address AI companion apps, which have quickly gained millions of users, including teens. In September, Character.ai users spent an average of 93 minutes a day on the app — 18 minutes longer than the average time spent on TikTok, according to Sensor Tower, a market intelligence firm.
Character.ai and Google respond
Character.ai spokesperson Chelsea Harrison declined to comment on the lawsuits. In a statement, she said the company aims to create a “safe and engaging space” for users. Harrison added that Character.ai is developing a model specifically for teens and has improved its ability to detect and respond to issues like self-harm.
Google, which licenses Character.ai’s technology, is also named in the lawsuits. The plaintiffs allege Google supported the app’s development despite knowing about safety risks and benefited from user data collected from minors.
Google spokesperson José Castañeda denied the allegations, stating that Google and Character.ai are “completely separate companies” and emphasizing that “user safety is a top concern.”
