Dynamic Selection and Configuration of Black-box Optimization Algorithms

Introduction

Black-box optimization algorithms are among the most widely applied optimization techniques in practice, used to solve numerous problems across a broad range of industrial branches and academic disciplines every day. Given this importance, it is not surprising that a plethora of different black-box optimization algorithms exist, complementing each other in strengths and weaknesses.

Project Objectives

In the dynaBBO project, we set out to obtain more efficient black-box optimization techniques by leveraging this complementarity, both with respect to different problem instances and with respect to different stages of the optimization process. To this end, we will develop approaches that select and dynamically switch between different black-box optimization algorithms "on the fly".

Research Questions

The two key research questions that guide our project are:

1. When to switch from one algorithm to another?
2. How to warm-start the selected solver so that it can continue the search as effectively as possible?

Both questions are largely under-explored and are handled rather naively in practice.

Methodology

To obtain our dynamic approaches, we intertwine insights about black-box optimization algorithms, obtained through rigorous theoretical analyses, with automated machine learning techniques. In particular, we will design trajectory-based algorithm selection and configuration techniques that combine exploratory landscape analysis with newly designed algorithm features that capture information about the solver-instance interaction.

Comparison of Techniques

We compare the efficiency of these feature-based approaches with:

• Deep learning techniques
• Reinforcement learning
• Approaches based on hyperparameter optimization

Impact and Applications

We will further increase our project's impact by validating its results on applications in bio-medicine and in computational mechanics.