A classifier is calibrated when its confidence agrees with the fraction of predictions it gets right at that confidence. Guo et al. (2017) found that modern deep networks, unlike those of a decade earlier, were often overconfident, with predicted probabilities higher than their accuracy.
Concept 66 of 100CalibrationTheory · Study phase 8: Scaling, theory & multimodal
Question
Negative log-likelihood is a proper scoring loss, so its unrestricted minimum on the true data distribution is calibrated; why do networks trained to very low training loss still end up overconfident on test data (Guo et al., 2017)?
How to study this pagePredict · Check · Generalize · Continue
PredictSay what you expect.
Pick a concrete input and say what the equation should give before you read the math.
CheckCheck against the math and the papers.
Work through the math, then read the cited papers for the original argument.
GeneralizeName what stays the same.
Identify the part of the idea that survives a change of notation or model, so you can recognize it in another paper or system.
ContinueFollow the connections.
Use the prerequisites and related concepts at the end of the page to fill the next gap.
Why It Matters for Modern Models
A classifier is calibrated when its confidence agrees with the fraction of predictions it gets right at that confidence. Guo et al. (2017) found that modern deep networks, unlike those of a decade earlier, were often overconfident, with predicted probabilities higher than their accuracy.
Calibrated probabilities matter wherever a decision depends on how likely a prediction is to be right, for example when deciding which cases to pass to a person for review.
Language models can be calibrated as well: Kadavath et al. (2022) found larger models well calibrated on multiple-choice and true/false questions posed in a suitable format, and OpenAI (2023) reported that GPT-4's pretrained model was highly calibrated on such questions but less so after post-training.
Questions to Think About
Why can a single temperature, fitted after training, remove much of the miscalibration in Guo et al.'s experiments without changing a single prediction?
Guo et al. (2017) found deeper and wider networks less calibrated, while Minderer et al. (2021) found some of the most recent image models, notably those without convolutions, among the best calibrated; what explains the difference?
Core Math
Key Equation
P(Y=Y^∣P^=p)=p
A classifier is calibrated if, among the predictions it makes with confidence p, a fraction p are correct:
P(Y=Y^∣P^=p)=pfor all p∈[0,1]
where Y^ is the predicted class and P^ the probability the model gives it.
Expected calibration error (ECE) groups the n test predictions into M bins by confidence and averages the gap between accuracy and confidence in each bin:
ECE=m=1∑Mn∣Bm∣acc(Bm)−conf(Bm)
The estimate depends on the binning: Kumar et al. (2019) found that, measured more carefully, methods such as temperature scaling are less calibrated than binned estimates suggest.
Temperature scaling fits a single scalar T>0 on held-out data and divides every logit by it:
qi=∑jexp(zj/T)exp(zi/T)
T>1 softens the predictions and reduces overconfidence. Dividing all logits by the same T does not change which class is largest, so accuracy is unchanged (Guo et al., 2017).