Gradient Descent and its Variants| Towards AI

A Quick Guide to Gradient Descent and its Variants

And if you do not know where to start from, optimize the learning rate

In machine learning, one deals with some mathematical model z = F(x,θ) which is a function of some input variables x and a set of parameters θ. The model can be for example an artificial and possibly deep neural network. The name of the game is to find the set of parameters that minimize the error in predicting some targets y…