BMW’s AR windshield uses gaze detection to respond to passengers' needs

At CES on Tuesday, BMW unveiled what it thinks is the future of autonomous mobility. A concept augmented reality windshield works with gaze tracking and BMW’s Intelligent Personal Assistant for a decidedly decadent passenger experience. The gaze detection tracks what you see outside of the vehicle as well as focuses on certain information on the windshield.

The future has its eye on you — The windshield concept, the BMW i Interaction EASE, looks sleek and futuristic but promises human-like interaction. A transparent display is overlaid to provide driving information, including nearby signs, buildings, etc.

“The BMW i Interaction EASE demonstrates what mobility might feel like in the future once autonomous driving becomes commonplace: luxurious, human, and intuitive,” said Adrian van Hooydonk, Senior Vice President of BMW Group Design. “Passengers start their journey with the feeling of having already arrived.”

The core of this cozy ambiance is the windshield’s gaze detection. AI follows and interprets eye movement of passengers and the driver to contextualize variables like traffic or just satisfy a curiosity about a restaurant you passed by. Gaze detection would work in concert with an advanced version of BMW’s gestures and voice-activated Personal Assistant, requiring the power of 5G networks.

Three modes, Explore, Entertain, and Ease, allow riders to choose their experience. Explore brings your attention to the outside world while Entertain immerses passengers in a cinematic experience. Ease literally transforms the vehicle, slipping seats into a “zero-gravity” position, dimming the display, and making the windows opaque for comfortable resting.

These features aren’t that far away; some are slated to appear in the 2021 BMW iNEXT. The vehicle will support 5G and a more rudimentary AR display.