Before 16-year-old Adam Raine died as a result of suicide, spent months of consultation with Chatgpt about his plans to end his life. Now his parents are making the first known lawsuit against Openai, Novel York Times Reports.
Many AI chatbots addressed to consumers are programmed to activate the security functions if the user expresses the intention to hurt himself or others. But tests He showed that these safeguards are far from reliable.
In the case of Raine, using the paid version of ChatgPT-4O, and and often encouraged him to seek professional assist or contact with the assist line. However, he was able to bypass these handrails, saying Chatgpt, that he asked about the methods of suicide on the fictitious story he wrote.
Opeli turned to these shortcomings on his blog. “When the world adapts to this new technology, we feel deep responsibility for helping those who need it the most”, post reads. “We are constantly improving the way our models react to sensitive interactions.”
Despite this, the company confirmed the limitations of existing safety training for vast models. “Our safeguards work more reliably common, short exchanges,” continues post. “Over time, we learned that these protections may sometimes be less reliable in long interactions: as the model’s security of the model’s safety training increases, they can degrade.”
These problems are not unique to Openai. Character. Chatbots powered by LLM were also associated with cases of illusions related to AI, which existing security tried to detect.
TechCrunch event
San Francisco
|.
October 27-29 2025