EyeReal, a display system that enables glasses-free 3D experiences using AI, is announced
Until now, enjoying 3D movies and TV shows required special glasses and a display. However, a research team from the Shanghai Artificial Intelligence Laboratory and Fudan University has announced EyeReal , a system that uses AI to provide a 3D viewing experience with an unprecedentedly wide viewing angle, without the need for special glasses.
Glasses-free 3D display with ultrawide viewing range using deep learning | Nature
《Nature》: AI movement, a completely new system for optical information communication! Book: Realization of eyes “True naked eye 3D”, zero drilling, super angle, Makoto Gaoho - Shanghai Artificial Intelligence Realization Room
https://www.shlab.org.cn/news/5444217
The biggest hurdle in developing conventional naked-eye 3D displays has been the 'spatial bandwidth product (SBP)' of the optical system. Previous research has shown that there is a trade-off between the 'spatial extent' of the screen size and the 'bandwidth' of the image, which indicates the angle at which the image can be viewed. In other words, if you try to make the screen larger, the viewing angle narrows, and conversely, if you want to make it visible from a wider range, you have no choice but to make the screen smaller.
For example, holographic displays that realize 3D with the naked eye have a wide viewing angle but are limited to a few centimeters in size, while displays that show multiple lines of sight by diffusing light using vibrations or structures can be made larger, but the angles at which 3D images can be viewed are quite limited.
The newly developed EyeReal solves this problem by concentrating limited optical information 'where the human eye is,' rather than trying to overcome physical limitations. The research team adopted a relatively inexpensive hardware configuration consisting of multiple commercially available liquid crystal panels (LCDs) stacked together with a backlight and polarizer. This configuration is notable for being constructed using common consumer components, without requiring special lens arrays or expensive optical elements.
At the heart of the system is an algorithm that combines a physically accurate binocular vision model with deep learning. EyeReal uses a depth-sensing camera (RGB-D) to track the observer's pupil position in real time and generates an optimal light field for only the eye position and its surrounding area at that moment.
Specifically, a neural network analyzes the image on the retina and calculates how the multi-layered liquid crystal panel should modulate the phase of light based on the laws of physics. This processing is performed at a refresh rate of over 50Hz, so it can track the viewer's head movements without delay, ensuring that the correct 3D image is always delivered.
With this technology, EyeReal achieves both an ultra-wide field of view exceeding 100 degrees and a resolution of 1920 x 1080 pixels. Experiments were conducted using 3D content, including complex cityscapes and everyday objects, and observers experienced smooth, uninterrupted motion parallax regardless of horizontal, vertical, or forward/backward movement. Furthermore, the technology supports 'focus adjustment,' which reproduces blur according to focal length, and has been shown to reduce discomfort caused by misalignment of the eyes' focus and orientation, a problem with conventional VR/AR devices.
The Shanghai AI Laboratory believes this technology is also significant in the context of 'AI for Science.' They hope that EyeReal will become a new tool for researchers to intuitively and immersively handle data, such as visualizing the three-dimensional structure of galaxies in astrophysics or observing the dynamic changes of cells in biomedicine. Furthermore, they argued that this approach, which improves the efficiency of light field information utilization, could also serve as a data foundation for AI models to more accurately understand and simulate complex physical phenomena in the real world.
The research team also aims to further refine the AI system and upgrade it to support multiple users simultaneously. While EyeReal is optimized for a single viewer at the time of writing, by integrating technologies such as simultaneous split display and directional backlighting, it is expected that multiple users will be able to enjoy natural 3D images simultaneously from their own positions in the future.
Related Posts:
