Exam Professional Machine Learning Engineer topic 1 question 109 discussion

B larger learning rates can reduce training time but may lead to model oscillation and may miss the optimal model parameter values.

When you observe oscillations in the loss during training, it is often a sign that the learning rate is too high. A high learning rate can cause the optimizer to overshoot the minimum of the loss function

A. Decrease Batch Size: While a smaller batch size can sometimes help with convergence, it can also lead to slower training. It might not necessarily address the issue of oscillation. C. Increase Learning Rate: A higher learning rate can cause the loss to jump around more erratically, potentially worsening the oscillation problem. D. Increase Batch Size: A larger batch size can lead to smoother updates but might also make the model less sensitive to local gradients and hinder convergence, especially with an already oscillating loss.

I don't understand

Went with B

Answer B

having a large learning rate results in Instability or Oscillations. Thus, the first solution is to tune the learning rate by gradually decreasing it. https://towardsdatascience.com/8-common-pitfalls-in-neural-network-training-workarounds-for-them-7d3de51763ad

https://ai.stackexchange.com/questions/14079/what-could-an-oscillating-training-loss-curve-represent#:~:text=Try%20lowering%20the%20learning%20rate,step%20and%20overshoot%20it%20again.