Interactions with Future Reach-Through Volumetric Displays

Introduction

Displays in the shape of televisions, computer screens, or phones are ever-present in our education, work, and entertainment. However, they do not take full advantage of our inner spatial abilities that we have to interact with the real world. True 3D displays can provide the same visual cues as the real world without forcing the users to wear devices.

Current Limitations

With state-of-the-art (SoA) displays, the users cannot reach inside the display volume to directly interact with the virtual objects as they would do in real life. We envision a volumetric display capable of projecting true 3D virtual objects in mid-air that can be reached by the users to enable direct interaction, i.e., a reach-through volumetric display (RVD). This vision has been presented in multiple movies and books, but there is no realization.

Proposed Technologies

Three novel technologies will be developed and combined to create an RVD:

1. Fast time-multiplexed acoustic fields will create virtual force fields that give shape to microfabricated light-scattering particles.
2. Tomographic illumination will shine on the particles as a more scalable alternative to phase-based holography.
3. Volumetric tracking of the particle distribution will control the previous technologies in a closed-loop manner.

Applications and Framework

Applications will serve as benchmarks to test novel interaction techniques and develop a framework that fills in the knowledge gap for interactions with as yet nonexistent RVDs.

Project Objectives

The objectives of the project are:

• O1) Find a set of technologies that enables the realization of RVDs.
• O2) Create interaction techniques for RVDs and categorize them using a framework that will be applicable to future displays; our current frameworks for 3D interactions may not be applicable to RVDs.
• O3) Enhance an RVD with tactile sensations, spatial audio, and study its effects on humans.

Qualifications

The PI is uniquely qualified with experience in designing mid-air interactions and using levitated particles for displays.