Cosine Learning rate decay

In this post, I will show my learning rate decay implementation on Tensorflow Keras based on the cosine function.

One of the most difficult parameters to set while training any deep learning model is the learning rate. If it is a big value the weights of the model will begin to oscillate and they will have big changes, preventing the model from adjusting to the changes in the error and if the learning rate is too…