April 29, 2026 by Ingrid Fadelli, Phys.org
Collected at: https://techxplore.com/news/2026-04-brain-approach-ai-overconfidence.html
Most contemporary artificial intelligence (AI) systems learn to complete tasks via machine learning and deep learning. Machine learning is a computational approach that allows models to uncover patterns in data that are useful for making predictions. Deep learning, on the other hand, is a subset of machine learning that entails the use of multi-layered neural networks, which can autonomously extract features and learn complex patterns from unstructured data, sometimes with little or no human supervision.
Many AI systems trained with these approaches also produce confidence scores for their predictions. These scores are essentially estimates of how probable it is for a specific prediction to be accurate. Past studies suggest that in many cases, AI systems are overconfident and assign high confidence scores to wrong answers, or even present inaccurate information as a fact. This limits their reliability, particularly in high-stakes applications where wrong predictions can have serious consequences.
Researchers at the Korea Advanced Institute of Science and Technology recently introduced a new brain-inspired training approach that could yield more realistic AI confidence estimates. Their proposed strategy, introduced in a paper published in Nature Machine Intelligence, entails briefly training artificial neural networks on random noise (i.e., data with no meaningful patterns) and arbitrary outputs, so that they can learn to produce more realistic confidence estimates before learning specific tasks.
“Uncertainty calibration, the alignment of predictive confidence with accuracy, is essential for the reliable deployment of machine learning systems in real-world applications,” wrote Jeonghwan Cheon and Se-Bum Paik in their paper. “However, current models often fail to achieve this goal, generating responses that are overconfident, inaccurate or even fabricated. We show that the widely adopted initialization method in deep learning—long regarded as standard practice—is, in fact, a primary source of overconfidence.”
Aligning AI confidence with the accuracy of predictions
A mismatch between the accuracy of an AI system’s predictions and how confident it is in its predictions can be highly problematic. When it comes to AI-based diagnostic tools meant to be deployed in medical settings, this mismatch could result in misdiagnoses. In the context of self-driving vehicles, it could result in accidents and collisions with other vehicles.
“To address this problem, we introduce a neurodevelopment-inspired warmup strategy that inherently resolves uncertainty-related issues without requiring pre- or post-processing,” wrote the authors. “In our approach, networks are first briefly trained on random noise and random labels before being exposed to real data. This warmup phase yields optimal calibration, ensuring that confidence remains well aligned with accuracy throughout subsequent training.”
The researchers’ approach entails adding a short warmup training stage before a model is fed real and task-specific data. During this stage, an AI model is presented with entirely random data and outputs or answers that are unrelated to this data. Once this initial training is complete, the model is trained normally on datasets that are relevant to the task it is learning to complete.
The researchers trained a model using their proposed approach and then compared its performance to that of models trained using standard machine learning methods. Their findings were very promising, as models who completed the warmup training appeared to be less prone to overconfidence and produced lower confidence scores for incorrect predictions, but appropriate confidence scores when their responses were correct.
“The resulting networks demonstrate high proficiency in the identification of ‘unknown’ inputs, providing a robust solution for uncertainty calibration in both in-distribution and out-of-distribution contexts,” wrote Cheon and Paik.
A brain-inspired training strategy for better predictions
In the future, the strategy devised by this research team could be refined further and applied to a broader range of AI models. This could help to further assess its potential across a wider range of real-world AI applications.
A key advantage of the new approach is that it does not require any complex engineering or additional processing steps, such as the post-processing of training datasets. Instead, it merely requires the introduction of a short but effective pre-learning warmup session.
The recent work by Cheon and Paik could eventually contribute to the development of safer and more reliable AI systems that are better at estimating the probability that their predictions are accurate. This could in turn facilitate the deployment of AI-based tools in contexts where wrong predictions can have serious repercussions, such as clinical settings or in other high-stakes scenarios.
Publication details
Jeonghwan Cheon et al, Brain-inspired warm-up training with random noise for uncertainty calibration, Nature Machine Intelligence (2026). DOI: 10.1038/s42256-026-01215-x
Journal information: Nature Machine Intelligence
Leave a Reply