Artificial intelligence (AI) and extended reality (XR): Two technological advancements that bring reality to a vast number of individuals.

Our guest author explains how the revolutionary combination of AI and XR is fueling a new era of Mixed Reality, with increased accessibility and advanced capabilities for highly engaging virtual experiences.

Author: David Weinstein, Director of Virtual and Augmented Reality, NVIDIA

The leading companies in Virtual Reality are releasing their newest headsets for Mixed Reality, focusing on creating smaller and lighter designs that offer impressive capabilities for spatial computing. With the continuous growth of the Extended Reality (XR) ecosystem, a wider range of industries and users are adopting XR technologies, which are transforming the way people work and engage in recreational activities.

Companies ranging from Google to Lowe’s have begun employing Mixed Reality headsets to enable consumers to play games, retail employees to collaborate inside immersive 3D models of stores, and healthcare professionals to model the human body for drug discovery and improved surgical planning.

While the movement to adopt Virtual Reality across business- and consumer-focused organizations isn’t new, only now are advancements in AI — such as computer vision, speech AI, and AI assistants — allowing lightweight devices to deliver photorealistic immersion, interactivity, and seamless Mixed Reality.

Another common denominator for the expansion of Mixed Reality is the OpenXR standard, which lets developers build 3D applications using a common application programming interface to run across a wide range of Extended Reality (XR) devices without the need to add and maintain expensive customizations. This new standard is being rapidly adopted across the ecosystem by HTC, Microsoft, Varjo and dozens more, resulting in faster product development cycles and a flurry of new applications.

The Dynamic Duo: AI and XR

AI and XR are closely linked as the industry aims to transition from selling a limited number of 20 million devices to ensuring that every adult in a household has at least one device. AI plays a crucial role in speeding up the rendering process, enabling the attainment of high resolution and frame rates necessary for a smooth and immersive experience. Across the globe, businesses are using Augmented Reality to identify and monitor real-world objects, offering a user-friendly and accessible platform for engaging with virtual content.

This includes Medicalholodeck, a platform for medical collaboration and teamwork in Virtual Reality. It uses integrated AI and GPU power to stream 3D medical-imaging content to untethered headsets. With the new cloud rendering technology, capabilities like presurgical planning, real-time reconstruction of CT and MRI scans, and medical training on an XR headset are now at doctors’ fingertips.

Hopping into these immersive experiences, surgeons can better prepare for operations by minimizing the time and effort required to import or change hospital environments before beginning intense procedures. In addition, teams can interact and collaborate virtually for training and alignment on surgical procedures. The combined power of AI and XR is set to transform medicine by streamlining training and surgery preparation for global health professionals.

In the consumer realm, everyday activities like watching a nightly sports game may never be the same. 5G Edge-XR, a project led by London-based GPU-accelerated products and services company The GRID Factory, is bringing XR streaming to the sports broadcasting industry.

The sports-viewing experience has been revolutionized by 3D streaming, which allows game stats and player data to be displayed directly on the matches themselves, accessible on mobile devices. Fans now can watch live-action replays from different angles using smartphones, tablets, or headsets. This innovation is not limited to sports games, as businesses are now reimagining other viewing experiences as well.

A new look, inside and out

Based on its initial stage, VR might not have seemed ready to become popular. This was because the headsets were big, heavy, and connected to large workstations with long cables. As a result, users were limited to specific places and the costs were relatively high. Because of these restrictions, VR was not widely adopted until the modern mobile headset was introduced. Now, a wider range of users can enjoy better comfort and visual attractiveness.

Today’s headsets boast a lighter, sleeker look, with some offering unbelievably thin lenses. The shift from bulky to compact headset design has enabled consumers to interact in and with their environments for a longer time, without heavy strain on their necks. The shortening and integration of the connecting cable within the headset results in a cleaner design. Users can flaunt a more ergonomic, modernized look that’s as stylish as the new features within.

VR has undergone both external and internal redesign, resulting in improved features such as high-quality cameras and precise eye and body tracking. These enhancements provide users with immersive experiences and facilitate collaboration with individuals and teams worldwide. Additionally, AR and MR headsets are becoming more available, aiming to seamlessly merge the physical and digital realms. These headsets can serve as an extension of a user’s personal space and can be effortlessly incorporated into everyday activities.

Extending Reality for the Future

With AI and XR already acting as a powerful dynamic duo, the next big step will be defined by a wider distribution of stunning XR experiences that include AI assistants to help people create in the virtual world, as well as build and play in the real world. With the help of digital assistants, everyone will be able to tap into generative AI to spontaneously populate their virtual environments with complex 3D models, stunning, expansive environments, and even fully articulated, interactive 3D characters.

With generative AI, the opportunities for XR are limitless. The emerging trends of improved XR and powerful AI assistants will power digital experiences and extend to industries like manufacturing, virtual production, and more. From cutting the headset cord to taking VR to the cloud, huge strides have been made in the XR industry as its user base continues to expand. No longer constrained to a single dedicated space, users’ collective realities are expanding beyond what’s been previously possible. Whether using XR for e-commerce or investing in it for education, businesses can tap the newest developments in VR, AR and AI to stake their claim in the growing ecosystem.

David Weinstein is the senior director of Extended Reality (XR) at NVIDIA, responsible for managing the company’s XR products and SDKs, and for engaging with partners to deploy them worldwide. Before joining NVIDIA, Dave founded and ran three tech startup companies.

Please be aware that the articles may contain affiliate links to online stores. If you make a purchase through these links, MIXED will receive a commission from the provider. Rest assured, the price remains the same for you.