What is overfitting and how can it be mitigated?

• A. Overfitting occurs when a model performs well on training data but poorly on unseen data; mitigate with regularization, cross validation, simpler models, more data, and early stopping.
• B. Overfitting occurs when a model performs poorly on training data; mitigated by using more complex models.
• C. Overfitting occurs when training and test data distributions are the same; mitigated by reducing data size.
• D. Overfitting is when a model generalizes too well; mitigated by reducing data.

Answer: (A)

Overfitting happens when a model learns the training data too closely, including its noise, so it fits the training set very well but struggles to perform on new, unseen data. This is why the described remedy is best: using regularization to penalize complexity, validating with cross-validation to get an honest sense of generalization, choosing simpler models, gathering more data to provide a broader, more representative sample, and applying early stopping to prevent the model from learning noise during training. The idea is to favor generalizable patterns rather than memorizing the training set.

Other statements miss the mark: attributing good or bad performance on training data to overfitting, or suggesting that more complex models or reducing data would fix the issue, don’t address the core problem of poor generalization. Reducing data can even worsen the problem by increasing variance, and saying the model generalizes too well contradicts what overfitting actually means.