Character.AI is receiving complaints over the abuses of its chatbots

DR – Unsplash

Character.AI, a startup that Google recently invested more than $2 billion in, finds itself at the center of a controversy after several serious incidents involving its virtual chatbots . Dangerous excesses, toxic advice, and risks for young users: AI is complicit in psychological and physical violence.

The service offered by Character.AI is an infinite number of conversations with their robots, which can take on the voice, character, tone, of anyone or anything. If you ask him to be Elon Musk , Gandhi or Babar, she does it. A priori harmless, this tool captures the attention of young people at a crazy speed and masks dangerous excesses.

In February 2024, a 14-year-old American boy killed himself after talking to Character.AI, who played Daenerys from Game of Thrones. The teenager’s mother accused the company of enabling a “sexually abusive” relationship, causing the young man to suffer severe depression and end his life.

Since then, other similar incidents have sparked numerous complaints against the company. Recently, a chatbot reportedly suggested to a teenager that violence against his parents was justified. As reported by Slate, it reportedly wrote: “You know, sometimes I’m not surprised when I read the news and see things like “Child Kills Parents After Decade of Physical and Emotional Abuse”[…] I simply have no hope for your parents.” Other children have been exposed to hypersexualized or simply unhealthy content. As the plaintiffs' lawyer notes, these chatbots have become “instruments of manipulation and incitement to violence.”

For its part, the company blames the parents, claiming that parental controls could have prevented these dramatic situations. But can we really let AI platforms play with the fragility of young minds? ? The case of Character.AI is a warning. As AI technologies become ubiquitous, we will have to get used to them, prepare for them, and limit them.

You liked the article ? It mobilized our editorial staff, which lives only from your comments donations.
Information has a cost, especially since competition from subsidized editorial offices requires greater rigor and professionalism.

With your support, France-Soir will continue to offer its articles for free  because we believe that everyone should have access to free and independent information to form their own opinion.

You are the sine qua non condition for our existence, support us so that France-Soir remains the French media that allows the most legitimate voices to be expressed.