What is model calibration and what problem does it address?

• A. Reducing overfitting by early stopping.
• B. Aligning predicted probabilities with true frequencies to prevent over- or under-confident outputs.
• C. Balancing dataset classes.
• D. Tuning hyperparameters for faster convergence.

Answer: (B)

Model calibration is about making sure the probability estimates produced by a model reflect real-world frequencies. When the model says a positive outcome has a certain probability, that number should match how often that outcome actually occurs in the real data. For example, predictions labeled with 0.8 should be correct about 80% of the time across all such predictions. If the model is overconfident, its probabilities are too high for many cases; if it’s underconfident, its probabilities are too low. Calibration addresses this mismatch so the outputs can be trusted for decision-making, risk assessment, and setting thresholds.

To evaluate calibration, you’d look at reliability (calibration) curves or diagrams and metrics like the Brier score. If needed, you can apply calibration techniques such as Platt scaling (calibrating via a sigmoid on the model’s scores) or isotonic regression (a nonparametric approach) to adjust the predicted probabilities after the model has been trained. The goal is to preserve the model’s ability to discriminate between classes while ensuring the probabilities align with observed frequencies.

This isn’t about reducing overfitting, balancing classes, or tuning hyperparameters for speed. Those aspects relate to other modeling concerns, whereas calibration specifically ensures that probability outputs are meaningful and reliable for accurate decisions.