New Frontiers in Projection-Free Methods for Continuous Optimization

Introduction

Efficient algorithms for continuous and in particular convex optimization have revolutionized science and engineering in the past decades, providing the engine that drives numerous key technical and computational practices used across almost every scientific and engineering discipline. In particular, it is one of the main pillars of the ongoing data science and AI revolution.

Projection-Free Methods

For many important large scale optimization problems that include constraints that are complex yet highly-structured, the algorithmic weapon of choice are so-called projection-free methods, which are mostly based on the classical Frank-Wolfe method. Despite vast interest and progress in recent years on scaling up projection-free methods, their optimization oracle complexity remains significantly inferior to their projection-based counterparts.

Challenges in Optimization

This shortcoming prevails throughout almost every optimization paradigm of interest and is often the bottleneck in further scaling up this important family of optimization methods.

Project Goals

The overarching goal of this project is the foundational and systematic study of the landscape of continuous optimization with computationally efficient (projection-free) optimization oracles.

Research Objectives

My goal is to develop an understanding, across a variety of central optimization paradigms, of how far we can push methods that rely only on simple and efficient optimization steps towards matching the complexities of state-of-the-art projection-based methods (that rely on computationally-expensive optimization steps).

Expected Outcomes

I envision that the novel algorithmic and methodological results that will come out of this research will lay the foundations for a new generation of far more advanced algorithms for continuous optimization with structured constraints. These will allow tackling, on a practical scale, a much richer variety of optimization settings and will be built to leverage, in a more specialized fine-grained manner, favorable and plausible structure of optimization problems.