Overcoming the curse of dimensionality through nonlinear stochastic algorithms

Introduction

In a series of relevant real-world problems, it is of fundamental importance to approximatively compute evaluations of high-dimensional functions. Such high-dimensional approximation problems appear in various fields, including:

• Stochastic optimal control problems in operations research
• Supervised learning problems
• Financial engineering, where partial differential equations (PDEs) and forward-backward stochastic differential equations (FBSDEs) are used to approximatively price financial products
• Nonlinear filtering problems, where stochastic PDEs are used to approximatively describe the state of a given physical system with only partial information available

Curse of Dimensionality

Standard approximation methods for such problems suffer from the so-called curse of dimensionality. This means that the number of computational operations of the approximation method grows at least exponentially in the problem dimension.

Project Objectives

The key objective of this project is to design and analyze approximation algorithms that provably overcome the curse of dimensionality in the following cases:

1. Stochastic optimal control problems
2. Nonlinear PDEs
3. Nonlinear FBSDEs
4. Certain SPDEs
5. Certain supervised learning problems

We intend to solve many of the above-named approximation problems by combining different types of multilevel Monte Carlo approximation methods, particularly:

• Multilevel Picard approximation methods
• Stochastic gradient descent (SGD) optimization methods

Additional Goals

Another chief objective of this project is to prove the conjecture that the SGD optimization method converges in the training of artificial neural networks (ANNs) with ReLU activation.

Expected Impact

We expect that the outcome of this project will have a significant impact on the way high-dimensional PDEs, FBSDEs, and stochastic optimal control problems are solved in engineering and operations research. Additionally, it will enhance the mathematical understanding of the training of ANNs through SGD optimization methods.