Gradient Descent: Into the Algorithm

An explanation for one of the most broadly used and important algorithms in Machine Learning

Credit

Objective of gradient descent and a brief dive down the cost curve:

Image credit
Image credit
Blue lines are the actual values, the red line is the predicted. Image credit

Batch Gradient Descent

Image credit

Stochastic Gradient Descent

Image credit

Mini-Batch Gradient Descent

Image credit
Image credit: Sahdev Kansal

Conclusion