Divergence and convergence in dialogue: The dynamic management of mismatches

Introduction

Human interaction is deceptively simple to engage in, yet surprisingly challenging to account for theoretically. Existing theories of language and cognition cannot fully account for the complex dynamics of verbal and non-verbal behaviours in interaction, which is becoming even more apparent with our increasing use of computer-mediated communication, such as the currently ubiquitous Zoom calls.

Vision of DivCon

With DivCon, my vision is to transform our basic understanding of human interaction by showing how successful dialogue is driven by incremental, local, and dynamic processes of mismatch management. In our everyday interactions, we continuously make predictions about what will happen next, based on how our own and others' behaviour affects the world, to open up new possible courses of action.

Predictions in Dialogue

In dialogue, these predictions are about:

1. Sounds
2. Words
3. Inferences
4. Non-speech actions such as gestures or eye gaze

If our expectations are not met, we have to ascertain if the mismatching input can be resolved or integrated as a surprising but rewarding outcome (as in the case of humour).

Data Production

DivCon will produce a suite of corpus and experimental data for exploring the timely issues of communication via different forms of computer-mediated communication, including:

• Text-based chats
• Video calls
• Virtual reality meetings

Experimental Platform

To do this, the project will create a novel experimental platform for experiments in real-time live multimodal interactions using avatars and virtual reality.

Theoretical Development

The formal arm of the project will develop a precise theory of divergence and convergence in interaction which unifies verbal and non-verbal dialogue phenomena including:

• Gesture
• Gaze
• Feedback
• Laughter

This will be done using core notions of prediction and underspecification.

Implementation in Conversational AI

This model will be implementable in conversational AI, an important step in the path to genuinely adaptive conversational AI systems, which are still beyond the reach of researchers despite the promise of recent decades.