Which of the following are common variants of gradient descent?

• A. Stochastic Gradient Descent (SGD), mini-batch gradient descent, and Adam.
• B. Principal Component Analysis, K-Means, and PCA.
• C. Cross-Validation, Regularization, and Dropout.
• D. Backpropagation, Activation functions, and Softmax.

Answer: (C)

Common variants of gradient descent include stochastic gradient descent, mini-batch gradient descent, and Adam. SGD updates parameters after each individual example, which makes learning fast on large datasets and introduces noise that can help explore the loss surface. Mini-batch gradient descent uses small random subsets of data for each update, balancing the instability of SGD with more stable progress. Adam is an optimizer that combines momentum with adaptive learning rates, adjusting the step size for each parameter based on estimates of past gradients, often yielding faster and more reliable convergence with less manual tuning.

The other options mix techniques that aren’t variants of gradient descent. Cross-Validation, Regularization, and Dropout relate to evaluating models or preventing overfitting rather than changing how gradients are computed or how updates are made. Unsupervised algorithms like PCA and K-Means aren’t gradient-descent variants, and components such as Backpropagation, Activation functions, and Softmax are parts of neural networks rather than different gradient-descent update methods.