Are bad incentives guilty of AI hallucinations?

AND New research article From OpenAI, he asks why enormous language models, such as GPT-5 and chatbots, such as chatgpt, still hallucicins and whether you can do something to reduce these hallucinations.

IN Blog post summarizing the articleOpeli defines hallucinations as “likely but false statements generated by language models” and admits that despite improvements, hallucinations “remain a fundamental challenge for all large language models” – those that will never be completely eliminated.

To illustrate, scientists say that when they asked “commonly used chatbot” about the title of Adam Tauman Kalai’s doctoral dissertation, they received three different answers, they are all wrong. (Kalai is one of the authors of the newspaper.) Then they asked about his birthday and received three different dates. Once again, everyone was wrong.

How can chatbot be so wrong – and sounds so confident that his evil? Scientists suggest that hallucinations are created, partly due to the preferring process, which focuses on the proper predictions of the next word, without real or false labels related to the training instructions: “The model sees only positive examples of liquid language and must bring general distribution.”

“Spelling and parentheses follow coherent patterns, so errors disappear with the scale,” they write. “But arbitrary facts of low frequency, like the animal’s birthday, cannot be foreseen, and therefore lead to hallucinations.”

The proposed solution of the article, however, is less focused on the initial initial process, and more on the assessments of enormous language models. He claims that the current assessment models themselves do not cause hallucinations, but “determine bad incentives.”

Scientists compare these grades with types of multiple choice tests in which random guessing makes sense, because “you may be lucky and right”, leaving the empty answer “guarantees zero”.

“In the same way, when models are rated only in terms of accuracy, the percentage of questions that they have exactly the right one are encouraged to guess, not saying” I don’t know “.

The proposed solution is therefore similar to tests (such as SAT), which include “negative [scoring] To get bad answers or partial recognition for leaving empty questions to discourage blind guessing. “Similarly, Opeli claims that the model’s assessments must” punish some errors more than punishing uncertainty and give partial recognition for an adequate expression of uncertainty. “

And scientists say that it is not enough to introduce “a few new tests of aware of uncertainty” on the side. Instead, “commonly used evolution based on accuracy must be updated so that their scoring is discouraged by guessing.”

“If the main boards of the results are satisfying happy guesses, the models will learn to guess,” scientists say.