Facebook Reality Labs (FRL) is committed to building a future where the real and virtual worlds seamlessly blend, enhancing our daily experiences, increasing efficiency, and strengthening connections. One of the challenges in achieving this vision is battery life. To create virtual reality headsets and augmented reality glasses that can be comfortably worn for long periods of time (including a whole day), FRL must optimize device power consumption. As a step towards building the next generation of AR/VR systems, the lab is working on developing a graphics system that significantly reduces power consumption without compromising image quality.
DeepFovea is one of the neural network-based methods developed by FRL to address this challenge. This rendering system utilizes the concept of Generative Adversarial Networks (GANs), a recent invention in artificial intelligence, to simulate peripheral vision in everyday life. DeepFovea can reduce the computational resources required for rendering by up to 10-14 times without any noticeable difference in image quality. It surpasses the traditional foveated rendering system found in current Oculus products and can generate images that are perceptually identical to full-resolution images while rendering less than 10% of the pixels.
FRL has released a complete demonstration of the DeepFovea repository to help the graphics research community explore advanced perceptual rendering. The ultimate goal of FRL is to bring real-time foveated rendering to lightweight and highly efficient AR/VR devices that can be worn all day. DeepFovea sets a new standard for perceptual rendering efficiency and represents an important step towards this goal. The method is hardware-agnostic and compatible with various AR/VR research systems.
Though DeepFovea provides an important approach for efficient rendering in AR and VR, it is just the beginning of exploring ultra-low-power perceptual rendering. FRL has also released the DeepFovea demonstration content to contribute to the progress of perceptual and neural rendering technology in the graphics and vision science research community.