GRADUATE STUDENT SEMINAR: Implicit Biases of Stochastic Gradient Descent, Alexandru Damian

Graduate Student Seminars
Oct 25, 2022
12:30 pm
Fine Hall 214

Implicit Biases of Stochastic Gradient Descent

Abstract: Deep Learning relies on the ability of stochastic gradient descent (SGD) to navigate high-dimensional non-convex loss landscapes and return minimizers which generalize to unseen data. However, this process remains poorly understood. I will present two recent results which attempt to explain the generalization ability of SGD by proving that SGD has a strong preference for "flatter" minimizers which generalize better.