Reparametrizing gradient descent

Add to your list(s) Download to your calendar using vCal

• David Sprunger, University of Birmingham
• Friday 12 February 2021, 16:00-17:00
• Zoom link at the bottom of the abstract.

If you have a question about this talk, please contact Dan Ghica.

abstract: In this talk, we propose an optimization algorithm which we call norm-adapted gradient descent. We will start with an overview of the main optimization algorithms used in machine learning, then describe norm-adapted descent, and present experimental evidence to its efficacy. The new algorithm is similar to other gradient-based optimization algorithms like Adam or Adagrad in that it adapts the learning rate of stochastic gradient descent at each iteration. However, rather than using statistical properties of observed gradients, norm-adapted gradient descent relies on a first-order estimate of the effect of a standard gradient descent update step, much like the Newton-Raphson method in many dimensions. Based on the experimental results, it appears norm-adapted descent is particularly strong in regression settings but is also capable of training classifiers.

Join Zoom Meeting https://bham-ac-uk.zoom.us/j/87640578230?pwd=L0ZWcm9EZEcvRmNYVmxSRFFlTW5DQT09

Meeting ID: 876 4057 8230 Passcode: 280754

This talk is part of the Theoretical computer science seminar series.

This talk is included in these lists:

• Computer Science Departmental Series
• Computer Science Distinguished Seminars
• Theoretical computer science seminar
• Zoom link at the bottom of the abstract
• computer sience

Note that ex-directory lists are not shown.