Structured Interactive Perception and Learning for Holistic Robotic Embodied Intelligence

Introduction

Robot learning has made remarkable strides thanks to high-capacity neural models and extensive datasets. However, there are persisting research questions concerning large-scale robot learning models:

1. Are massive architectures and data needed for achieving robotic embodied intelligence to solve tasks intuitive to humans?
2. How can we make substantial progress toward robust and adaptive robot learning systems to operate in the dynamic real world?

I posit that these open problems stem from overlooking the underlying principles and structure that govern the intricate robot-environment interaction and evolution.

SIREN Overview

SIREN addresses these pressing issues by proposing a unique systemic view of robot learning through the holistic representation of robot and environment as an integrated system.

Action-Perception Cycle

To achieve this, we will unveil key properties of the action-perception cycle for developing embodied intelligence by studying the intertwined flow of information and energy within the components of the holistic system.

Proposed Framework

For that, we propose a framework that pioneers information-driven and physics-aware objectives that encompass the learning from embodied multisensorial streams of a modular graph representation of the robot-environment system and its dynamics. This framework is backed by the versatility of graph neural networks, allowing for modular uncertainty estimation to promote robustness.

Training Dynamics

Eventually, we will yield resilient dynamics for training uncertainty-aware, composable skills to adapt to new tasks.

Impact of SIREN

SIREN's breakthroughs will enable robots, like humanoid mobile manipulators, to merge in unstructured, human-like settings and perform challenging tasks that require smooth and efficient perception-action coordination, balancing generalization and robustness in the face of inevitable real-world uncertainties.

Future Research

Our paradigm shift opens avenues for future groundbreaking research rooted in SIREN's impacts toward continuous robot learning systems that are integrated and evolve with their environment.