Meta’s latest AI demonstration shows how the technology can help AR glasses recognize objects around you without depth sensors.
Meta SceneScript
Meta AI posted a video demonstrating how its Project Aria AR glasses can capture a point cloud that the SceneScript model based on Llama AI can interpret as identifiable real-world objects, like walls, windows, doors, and furniture.
The result is a formatted text file written in English language using structured markup code that includes the dimensions of the objects. The provided information is sufficient to outline the objects or generate a 3D representation by utilizing bounding boxes.
A Meta Quest 3 ran the SceneScript software and researchers used the VR headset’s passthrough view to show the overlaid point cloud data and labeled scene elements.
SceneScript has the ability to generate intricate geometries, enhancing the level of detail in objects such as tables, office chairs, and sofas.
A preview of what’s coming
While Meta’s Project Aria glasses capture data to reconstruct the scene, the Qualcomm Snapdragon 835 processor in these prototype devices isn’t sufficient for processing.
Project Aria doesn’t contain a depth sensor, so the point clouds are based solely on visual processing. That’s important for AR glasses since every sensor adds weight, which is major concern for something that rests on the bridge of your nose.
SceneScript isn’t available for consumers to use on the Quest 3 or Ray-Ban Meta Smart Glasses, but Meta’s research provides an intriguing look at what will be possible with upcoming AR glasses.
In the future, immersive augmented reality experiences can be enhanced by identifying different elements within a room. This allows virtual content to seamlessly interact with your surroundings. For instance, an application such as First Encounters on AR glasses like Quest 3 may achieve this level of interaction.
