GRADUATE STUDENT SEMINAR: Implicit Biases of Stochastic Gradient Descent, Alexandru Damian

Tue, Oct 25, 2022, 12:30 pm

Implicit Biases of Stochastic Gradient Descent

Abstract: Deep Learning relies on the ability of stochastic gradient descent (SGD) to navigate high-dimensional non-convex loss landscapes and return minimizers which generalize to unseen data. However, this process remains poorly understood. I will present two recent results which attempt to explain the generalization ability of SGD by proving that SGD has a strong preference for "flatter" minimizers which generalize better.

Fine Hall 214
Event category: 

Upcoming Events

PACM Colloquium, Prof. Oanh Nguyen, Brown University

Mon, Apr 3, 2023, 4:30 pm
Location: 214 Fine Hall


Tue, Apr 4, 2023, 11:00 am
Location: 214 Fine Hall

ANALYSIS OF FLUIDS AND RELATED TOPICS: Enhanced dissipation for time-periodic alternating shear flows; Kyle Liss, Duke University