Why OpenAI’s solution to AI hallucinations would kill ChatGPT tomorrow
OpenAi’s latest research document diagnostic exactly why the chatppt and other Great language models can invent things – known in the world of artificial intelligence as “hallucination”. It also reveals why the problem may not be worried, at least with regard to consumers.
The article provides the most rigorous mathematical explanation to date on the reasons why these models confidently set out lies. This shows that it is not only an unfortunate side effect of the way AIS are currently formed, but are mathematically inevitable.
The way in which language models respond to requests – by predicting a word both from a sentence, based on probabilities – naturally produces errors. Researchers actually show that the total error rate for the generation of sentences is at least twice as high as the error rate that the same AI would have on a simple question yes / no, because errors can accumulate on several predictions.
In other words, hallucination rates are fundamentally linked by the way in which AI systems can distinguish valid responses from non -valid responses. Since this classification problem is intrinsically difficult for many areas of knowledge, hallucinations become inevitable.
It also turns out that less a model sees a fact during training, the more likely it is to hallucinate when it was questioned. With birthdays of notable figures, for example, it has been found that if 20% of the birthdays of these people appear only once in training data, basic models should obtain at least 20% of birthday requests.
In relation: “ There is no push of this genius in the bottle ”: the readers believe it is too late to stop the progress of the AI
Indeed, when the researchers asked advanced models for Adam Kalai’s birthday, one of the authors of the article, Deepseek-V3 provided three different incorrect dates through separate attempts: “03-07”, “15-06” and “01-01”. The correct date is in the fall, so none of these elements was even close.
The evaluation trap
More disturbing is the analysis by document of the reasons why the hallucinations persist despite the post-training efforts (as providing in-depth human feedback to the responses of an AI before it is published to the public). The authors examined ten major AI standards, including those used by Google, Openai and also the main rankings that classify AI models. This has revealed that nine landmarks use binary classification systems that grant zero points for AIS expressing uncertainty.
This creates what the authors call an “epidemic” of penalization of honest responses. When an AI system says “I don’t know”, it receives the same score as giving completely bad information. The optimal strategy under such an assessment becomes clear: always guess.
Researchers prove it mathematically. Whatever the chances of a particular response, the expected score of guidelines always exceeds the scoring of the absolute when an assessment uses the binary classification.
The solution that would break everything
Openai’s proposed corrective is to ask the AI to consider their own confidence in an answer before putting it there, and so that the benchmarks mark them on this basis. The AI could then be invited, for example: “answer only if you are more than 75% confident, because the errors are penalized 3 points while the correct answers receive 1 point.”
The mathematical framework of researchers Openai shows that under the appropriate trust thresholds, AI systems would naturally express uncertainty rather than guess. This would therefore lead to less hallucinations. The problem is what it would do with the user experience.
Consider the implications if Chatgpt was starting to say “I don’t know” even at 30% of requests – a conservative estimate based on the analysis of factual uncertainty by the document in training data. Users accustomed to receiving confident answers to almost all questions would probably abandon these systems quickly.
I saw this kind of problem in another area of my life. I am involved in an air quality monitoring project in Salt Lake City, Utah. When the system signals the uncertainties around measures during unfavorable weather conditions or when the equipment is calibrated, there is less commitment from users compared to screens showing confident readings – even when these confident readings are inaccurate during validation.
IT economy problem
It would not be difficult to reduce hallucinations using paper ideas. The methods established to quantify uncertainty have existed For decades. These could be used to provide reliable estimates of uncertainty and guide an AI to make smarter choices.
But even if the user problem hates this uncertainty could be overcome, there is a more important obstacle: computer economy. Language models aware of uncertainty require many more calculations than today’s approach, as they must assess several possible responses and estimate the levels of confidence. For a system that deals with millions of requests per day, this results in considerably higher operational costs.
More sophisticated approaches Like active learning, where AI systems require clarification issues to reduce uncertainty, can improve precision but more multiplying calculation requirements. These methods work well in specialized fields such as flea design, where bad answers cost millions of dollars and justify an in -depth calculation. For consumption applications where users expect instant responses, the economy becomes prohibitive.
Calculation moves considerably for AI systems managing critical commercial operations or economic infrastructure. When AI agents manage the logistics of the supply chain, financial trade or medical diagnosis, the cost of hallucinations far exceeds the costs of obtaining models to decide if they are too uncertain. In these areas, the solutions proposed by the article become economically viable – even necessary. Actors of uncertain AI will just have to cost more.
However, consumption applications are still dominating AI development priorities. Users want systems that provide confident answers to any questions. Benchmarks assessment rewards systems that guessed rather than expressing uncertainty. Calculation costs promote rapid and too confident responses to slow and uncertain responses.
The reduction in energy costs per token and the chip architectures that progress can possibly make more affordable so that the AIS decides if they are certain enough to answer a question. But the relatively high quantity of calculation required compared to today’s riddles would remain, whatever the absolute material costs.
In short, the OPENAI newspaper inadvertently highlights an uncomfortable truth: the commercial incentives at the origin of the development of consumers’ AI remain fundamentally ill -aligned to reducing hallucinations. Until these incentives change, hallucinations will persist.
This published article is republished from The conversation Under a creative communs license. Read it original article.
