From A to B: Generalizing the mathematics of artificial neural networks (ANNs) to biological neural networks (BNNs)

Introduction

Why does the brain outperform AI? Artificial neural networks (ANNs) are at the core of the AI revolution. In the past years, enormous efforts have been made to unravel their mathematical properties, leading to fundamental insights and mathematical guarantees on when and why deep learning works well.

Differences Between ANNs and BNNs

ANNs are inspired by biological neural networks (BNNs) but differ in many respects:

• ANNs represent functions while BNNs represent stochastic processes.
• The gradient-based deep learning applied for ANNs is very different from the local updating of BNNs.

Superiority of BNNs

BNNs are superior to ANNs in the sense that the brain learns faster and generalizes better. Despite the urgency for answers and the rich and interesting mathematical structures that BNNs create, scarcely any theoretical attempts have been made to understand learning in the brain.

Need for Mathematical Statistics Approach

The stochastic process structure of BNNs and the need to understand the statistical convergence behavior call for a mathematical statistics approach. This project proposes the development of advanced mathematical tools in nonparametric and high-dimensional statistics to analyze learning in BNNs as a statistical method.

Novel Interpretation of BNN Parameters

The starting point is a novel interpretation of the local updating of BNN parameters as a specific and non-standard, derivative-free optimization method. Whereas derivative-free optimization is thought to be slow, our conjecture is that it leads to favorable statistical properties in the setting underlying BNNs.

Potential Impact of the Research

If the research is successful, it has the potential to:

1. Open a new research area in mathematical statistics.
2. Provide insights into how the brain learns.
3. Lead to recommendations on how to make AI more efficient with less training data.
4. Inform the training of neuromorphic computer chips mimicking BNNs.