AI Improves Your Results and Destroys Your Judgment
AI makes you perform better. But a series of studies shows it simultaneously degrades your ability to know when it is wrong.
A study published in Computers in Human Behavior in 2025 had 246 people complete a logical reasoning test -- the kind found in US law school entrance exams. Half used ChatGPT. Half did not.
Result: those who used AI scored higher. +3 points on average. AI works, no surprise there.
But then the researchers asked a different question: "How do you think you did?" Something strange emerged. AI users did not just overestimate their score. They overestimated it by an additional 4 points.
You get better. You think you got even better than you actually did. And in between, there is a cognitive blind spot that AI is quietly widening.
What We Mean by Metacognition
Metacognition is the ability to monitor your own cognitive processes. In plain terms: knowing what you know, knowing what you do not know. And above all, knowing when you are wrong.
It is an internal correction circuit. When you solve a problem and something feels off, that is your metacognition pulling the alarm. When you reread a sentence and realize it does not hold up, same thing.
That circuit is what AI is short-circuiting.
The Loop That Breaks
When you use AI to produce something -- an analysis, a piece of writing, a diagnosis -- you delegate the task. So far, so logical.
The problem: you also delegate the evaluation of the task. The cognitive feedback that normally tells you "that was hard" or "I was not sure about that" disappears. AI produces a smooth, confident, well-structured output. And you can no longer tell whether it is good because it is actually good, or because it looks good.
A systematic review published by Springer Nature in 2025 (35 studies analyzed) measured this at scale: 58.9% of users place blind trust in ChatGPT outputs without any verification. This is not laziness. It is a cognitive bias documented since the 1990s: automation bias.
The Advanced User Paradox
Here is perhaps the most counterintuitive finding: in that same study, participants with the highest AI literacy (those who best understand how these systems work) had even less accurate metacognition than beginners.
In other words: learning about AI does not protect you from the problem. In some cases, it makes it worse.
The likely explanation: the more comfortable you are with AI, the more credit you give it, and the less you trigger your internal alarm signal. Trust in the tool replaces vigilance. This is not about skill. It is a cognitive dynamic.
The Machines Do Not Know When They Are Wrong Either
What makes the situation particularly tricky is that the problem is symmetrical.
A study published on arXiv in 2025 evaluated 4 LLMs across 24,000 questions, measuring their calibration -- that is, their ability to know when their answers are reliable. Expected Calibration Error (ECE) measures the gap between the model's expressed confidence and its actual accuracy.
Kimi K2 showed an ECE of 0.726, with only 23.3% accuracy. The model expressed high confidence on massively incorrect answers. Claude Haiku 4.5 was better calibrated (ECE 0.122), but the problem exists across all models to varying degrees.
Result: you have a poorly calibrated human interacting with a poorly calibrated machine. The two dysfunctions reinforce each other.
When It Stops Being Theoretical
In radiology, 27 radiologists took part in a study published in Radiology in 2023. They were asked to read mammograms with AI assistance, but the AI sometimes gave false suggestions.
When the AI was wrong, novice radiologists fell below 20% accuracy. Experts dropped from 82% to 45.5%. Novice radiologists followed incorrect AI recommendations four times more often than experts (P = .009).
This is not anecdotal. It is a controlled trial, and the results show that automation bias takes hold even in trained healthcare professionals.
In insurance, the case is even more explicit. UnitedHealth used an algorithm called nH Predict to deny post-acute care to Medicare patients. Known error rate: 90% of denials are reversed on administrative appeal.
The algorithm was profitable precisely because only 0.2% of policyholders appeal. In February 2025, a US federal court refused to dismiss the case. The trial continues.
What You Can Do, Concretely
There is no perfect solution. But a few principles reduce the risk.
Add friction to high-stakes decisions. For important topics -- health, law, finance -- build in a systematic manual verification step, not an optional one. The fact that AI is fluent does not mean it is correct.
Ask AI about its limits, not just its answer. "What is your answer" is a less useful question than "what are the limits of your answer and where might you be wrong here?"
Recent models can partially identify their uncertainty zones if you ask directly.
Preserve tasks where you exercise your own judgment. Delegating everything to AI in a domain also means no longer training your internal correction circuit. Metacognitive skill, like any skill, atrophies without use.
This is not a call to use AI less. It is a reminder that trust in a tool should be proportional to your ability to verify its outputs -- and that this ability is precisely what AI can erode if you are not paying attention.
