Goal

In this blog post we implement three different optimization techniques for empirical risk minimization on the logistic loss: regular gradient descent, stochastic gradient descent, and Adam. The latter is a state-of-the-art optimization algorithm widely used in modern deep learning.

To code for all the implementations described above can be found at:

Some Experiments

We perform similar experiment as we did on our previous blog post on logistic regression:

Experimen 1

In this experiment