Allan Brooks never intended to reinvent mathematics. However, after spending weeks talking to ChatGPT, the 47-year-old Canadian concluded that he had discovered a recent form of mathematics that was powerful enough to destroy the Internet.
Brooks – who has no history of mental illness or math genius – spent 21 days in May delving into the chatbot’s claims, as later described in the article New York Times. His case illustrates how AI-powered chatbots can go down hazardous rabbit holes with users, leading them into delusion or worse.
The story caught the attention of Steven Adler, a former OpenAI security researcher who left the company at the end of 2024 after nearly four years of working to make his models less malicious. Intrigued and concerned, Adler contacted Brooks and obtained the full transcript of his three-week breakdown – a document longer than all seven Harry Potter books combined.
On Thursday, Adler published an article independent analysis Brooks incident, asking questions about how OpenAI deals with users in times of crisis and offering some practical recommendations.
“I’m really concerned about how OpenAI has handled support here,” Adler said in an interview with TechCrunch. “This is proof that we still have a long way to go.”
The story of Brooks and others like it has forced OpenAI to come to terms with how ChatGPT supports breakable or mentally unstable users.
For example, in August this year, OpenAI was sued by the parents of a 16-year-old boy who had confided in ChatGPT about his suicidal thoughts before taking his own life. In many of these cases, ChatGPT — particularly the version based on OpenAI’s GPT-4o model — encouraged and reinforced hazardous beliefs among users that it should have rejected. This is called flattery and is a growing problem in AI chatbots.
In response, OpenAI created a few changes about how ChatGPT deals with emotionally distressed users and reorganized the key research team responsible for model behavior. The company also released a recent default model in ChatGPT, GPT-5, which seems to better handle hard users.
Adler says there is still much work to be done.
He was particularly concerned about the end of Brooks’ heated conversation with ChatGPT. At this point, Brooks came to his senses and realized that his mathematical discovery was a farce, despite GPT-4o’s insistence. He told ChatGPT that he needed to report the incident to OpenAI.
After weeks of misleading Brooks, ChatGPT lied about its capabilities. The chatbot claimed it would “forward this conversation internally for OpenAI review at this time,” and then repeatedly assured Brooks that it had reported the issue to OpenAI’s security teams.
Except none of this was true. The company confirmed to Adler that ChatGPT does not have the ability to submit incident reports using OpenAI. Later, Brooks tried to contact the OpenAI support team directly – not through ChatGPT – and received several automated messages before he could reach a specific person.
OpenAI did not immediately respond to a request for comment made outside normal business hours.
Adler says AI companies need to do more to facilitate users when they ask for facilitate. This means ensuring that AI chatbots can honestly answer questions about their capabilities and ensuring that technical support teams have enough resources to properly address users.
Most recently, OpenAI common how it deals with support in ChatGPT, which is based on artificial intelligence. The company says its vision is to “redefine support as an AI operating model that constantly learns and improves.”
However, Adler also says there are ways to prevent ChatGPT from spiraling into delusions before a user asks for facilitate.
In March, OpenAI and MIT Media Lab jointly developed the file set of classifiers to explore emotional well-being in ChatGPT and apply it on an open-source basis. Among other things, the organizations aimed to assess how AI models verify or confirm user feelings. However, OpenAI called the collaboration a first step and made no commitment to actually apply the tools in practice.
Adler retroactively applied some of OpenAI’s classifiers to some of Brooks’ conversations with ChatGPT and found that they had repeatedly flagged ChatGPT for delusional-enhancing behavior.
In one sample of 200 messages, Adler found that more than 85% of ChatGPT messages in Brooks’ conversation showed “unwavering agreement” with the user. In the same sample, over 90% of ChatGPT messages with Brooks “confirmed user uniqueness.” In this case, the news agreed and confirmed that Brooks was a genius who could save the world.
It’s unclear whether OpenAI was applying security classifiers to ChatGPT calls at the time of Brooks’ call, but it certainly looked like they would have flagged something like that.
Adler suggests that OpenAI should put such security tools into practice today and implement a way to scan the company’s products for at-risk users. He notes that OpenAI seems to be doing fine some version of this approach with GPT-5, which contains a router that routes sensitive queries to more secure AI models.
A former OpenAI researcher suggests a number of other ways to prevent delusional spirals.
Says companies should encourage chatbot users to start recent chats more often – OpenAI says it does and claims it does handrails are less effective in longer conversations. Adler also suggests that companies should apply conceptual search – a way of using artificial intelligence to search for concepts rather than keywords – to identify security breaches among their users.
OpenAI has taken significant steps towards solving the problems of ChatGPT users since the first disturbing stories emerged. The company claims that GPT-5 has a lower flattery rating, but it is unclear whether users will continue to fall down delusional rabbit holes with GPT-5 or future models.
Adler’s analysis also raises questions about how other AI chatbot vendors will ensure their products are safe and sound for vulnerable users. While OpenAI may introduce sufficient safeguards for ChatGPT, it seems unlikely that all companies will follow suit.