Our See-Through 3D desktop finally public!

My intern Jinha Lee and I crunched some fun ideas during his research internship at Microsoft!




See-Through 3D desktop done by Jinha Lee and Cati Boulanger in the Applied Sciences Group

I thought it deserved a spot on my blog too! So our See-Through 3D Desktop offers a behind-the-screen interaction with a transparent OLED with view-dependent, depth-corrected gaze.

Despite advances in 3D sensing and display technologies, our interactions with computer desktops have remained stagnant from the form that evolved under 2D I/O modalities. Our See Through 3D desktop is a 3D spatial operating environment that allows the user to directly interact with her virtual desktop. The user can reach into the projected 3D output space with his/her hands to directly manipulate the windows.



Users can casually open up the See-Through 3D Desktop and Type on the keyboard or use a trackpad as in traditional 2D operating environment. Windows or files are perceived to be placed in a 3D space between a screen and the input plane.

The user can lift up her hands to reach the displayed windows and arrange them in this 3D space.A unique combination of a transparent display and 3D gesture detection algorithm collocates input space and 3D rendering without tethering or encumbering users with wearable devices. See-through 3D desktop is a term for the entire ensemble of necessary software hardware and design technological components for realizing this volumetric operating environment.

Voilà!