The Voices of VR Podcast

Hello! My name is Kent Bye, and welcome to the Voices of VR podcast. One of the trends with virtual and augmented reality is that it’s going to take all sorts of new information from our body — physiological data like heart rate, heart rate variability, eye tracking, visual attention, facial movements, and even what’s happening with our mouth. There’s a wealth of information that’s going to be fed into these immersive experiences.

One of the first virtual audio headsets that’s really trying to gather all this information and apply it to different enterprise training contexts is the HP G2 Reverb Omnicept Edition. This is a special edition of HP’s G2 Reverb, equipped with various sensors aimed at enabling specific independent software vendors (ISVs) to develop special applications that can leverage these sensors. I had a chance to try out the HP G2 Reverb Omnicept Edition and tested a couple of different software applications, observing how my behavior and actions translated into numerical data, such as a cognitive load metric that indicated how cognitively loaded I was during these experiences.

The challenge I found, at least initially, was that it’s sometimes difficult to correlate what these numbers conveyed with my own phenomenological experiences. This difficulty partly stems from the lack of real-time feedback crucial for close analysis, while another aspect is how HP processes the specific information into a calculated representation of cognitive load. I believe that, in the long run, there is significant potential in the direction this is heading.

I also had the privilege of talking to various people involved in managing, planning, and creating this platform, in addition to conducting experimental psychology, research, and science on specific applications using various forms of artificial intelligence. In this episode, I’ll chat with some of the folks from HP to unpack their intentions behind this product and share more about my direct experiences and the challenges involved in evaluating hardware that processes biometric information. I’ll also share some general thoughts about the HP G2 Reverb Omnicept Edition at the end.

Interview Segment

So this interview with Scott, Henry, and Erica took place on Friday, September 3, 2021. Let’s dive right in.

Scott Rawlings

Nice to see you again, Kent. I manage the Omnicept platform and I’m also working on some future platforms focused mostly on the software aspect in product management.

Henry Wong

Hey, Kent! I’m Henry Wong, a product manager in the virtuality group at HP, focusing more on our hardware and devices. I’m specifically the product manager for the HP Reverb G2 Omnicept Edition.

Erica Siegel

I’m Erica Siegel, I’m an experimental psychologist by training, and I’m a research scientist on this project. I work closely with Jeremy Bailenson, who I think you know well, and I’m the subject matter expert on the human inferences that are part of this product.

Exploring Backgrounds and Journeys into VR

Let’s each provide some context regarding our backgrounds and our journeys into VR as we work at HP and interface with the broader XR community. I’ll start. My background is in academic science. Before joining HP, I specialized in interpreting data from sensors to infer human psychology. I did a PhD in Boston and then became a research scientist at UC San Francisco focusing on stress physiology, developing a product with Samsung Research America to predict stress from blood pressure data. I was recruited to this team based on my scientific work and was excited to collaborate with cutting-edge machine learning scientists, HCI researchers, and academic researchers.

Now, my role at HP is to continue exploring the science behind human inference, ensuring that the methodologies we use are reliable and repeatable. It’s truly exciting work. We’re heavily engaged with several VR projects, making it a lot of fun!

Henry Wong

For my part, I started with HP in the commercial PC business before there was a VR organization. I was initially involved in VR when it was a nascent industry and was stunned by its potential to influence how we work and play. When I learned that HP was creating a dedicated business unit around VR, I jumped at the opportunity. I find it fascinating how people use VR on both the consumer and commercial sides, leading me to my current product management role. It’s an absolute joy to be developing a device— the Reverb G2 Omnicept Edition— that not only provides an immersive experience but integrates sensors to support the use cases Scott and Erica mentioned earlier. Looking to the future, we aim to make headsets even more immersive and to build capabilities for adapting experiences to various commercial contexts.

Erica Siegel

I began with flight simulation early in my career as a design engineer on a multimillion-dollar flight simulator, which laid the foundation for my interest in multimedia and nonlinear video solutions. Coming back to HP, I find VR fascinating as it revolves around creating experiences. It feels like my career is coming full circle, and it’s exciting to be part of this team.

Insights into the HP Omnicept Product

Could you provide more context regarding the HP Omnicept product and how it differs from something like the HP Reverb as a consumer device?

Scott Rawlings

Absolutely. The HP Reverb G2 Omnicept Edition is solely focused on commercial applications. The idea is to marry biometric sensors embedded in the VR headset with machine learning to understand the context of what a person is experiencing. This allows the application to dynamically adapt based on this real-time biometric feedback.

Henry Wong

One of the significant indicators we use is eye tracking— the movement of the eyes, pupil size, and gaze are crucial for interpreting visual attention. With these insights, we can feed data into machine learning models, which Erica can elaborate on further.

Erica Siegel

One of our key metrics is cognitive load, essentially a real-time measure of mental effort. We’ve found that cognitive load can be effectively estimated using physiological sensors, allowing us to predict how cognitively loaded a person feels while engaged in tasks. Over a two-and-a-half-year period, we collected data globally, allowing us to train a machine learning model capable of predicting cognitive load in real time.

Potential Applications and Insights

Henry Wong

One of the challenges with collaboration is that understanding cognitive load provides extra information beyond performance metrics. It’s essential that individuals can perform well while feeling comfortable, which leads to improved retention and overall well-being.

Erica Siegel

This concept plays a crucial role in training, where applications tell the trainer when a person is ready to move from VR training to hands-on work based on their cognitive load data. Additionally, in high-risk training, measuring cognitive load can help in elaborating decision-making processes during simulations.

It’s exciting to consider how VR can enable immersive experiences that build empathy and facilitate unique learning opportunities in various fields, from health to design. This spreads across many industries with the potential for significant impacts.

Looking Forward

As we continue to refine the HP G2 Reverb Omnicept Edition and the Omnicept platform, there’s much excitement around integrating more expressive capabilities, particularly for social VR. Our goal is to ensure these technologies empower users in those creative and collaborative spaces without unnecessarily restricting consumer use.

Erica, are there any emerging research opportunities that excite you with the development of Omnicept?

Erica Siegel

Absolutely! Omnicept opens doors for innovative research. With biofeedback applications, we can stream sensor data in real time, allowing developers to create dynamic environments that respond to individuals’ physiological states. This has vast potential for advancing rehabilitative techniques and enhancing experiences within various fields.

Conclusion

In summary, the HP G2 Reverb Omnicept Edition signifies a leap forward in utilizing biometrics within immersive environments, promising exciting developments across numerous sectors, from training to healthcare. I thank HP for providing me the opportunity to explore this technology and its applications. For anyone interested in learning more, check out hp.com/omnicept.

Thanks for listening to the Voices of VR podcast. If you enjoyed this episode, please share it with friends and consider supporting us on Patreon.