Less anxiety in VR: Pioneering work to reduce anxiety triggers in immersive media
In the era of social media, it is quite common for individuals in the 2D realm to come across various images randomly. This often involves an endless scrolling through the timeline, which may occasionally present undesired images. Examples include a close-up of a spider for those afraid of arachnids, a realistic war photograph depicting injured soldiers, or a detailed shot of fresh sushi accompanied by Kobe beef for individuals following a vegan lifestyle.
On a mobile device, tablet, or computer screen, you can shift your gaze away relatively fast and proceed to the following image. Therefore, although the encounter with a 2D fear stimulus may not have been enjoyable, it is easier to forget. Conversely, highly immersive 3D media aims to recreate real-life environments in virtual reality, allowing the viewer to feel fully immersed in the scene as if they were actually present. When it comes to 180° and 360° images, it becomes more challenging to divert your attention.
One possible approach is to shut one’s eyes and trust that the subsequent picture will not induce anxiety. Nonetheless, this response is typically not instinctively ingrained and may not be very beneficial, particularly for series of images with similar content. Due to the lifelike nature of virtual reality (VR), it is advisable to consider reducing the potential causes of anxiety in immersive media. This would enable a wider and more user-friendly utilization among diverse groups of users.
As developers of the VR image and slideshow viewer immerGallery, we have done just that. In collaboration with the University of Würzburg, student Amelie Hetterich has been working on the topic in her bachelor thesis. An abstract of the thesis was presented at the ICAT-EGVE scientific conference in Dublin in December 2023.
Since anxiety triggers can be very intense, especially in highly immersive, realistic VR media, this article presents research that reduces their influence and thus allows for a more comfortable, serendipitous viewing of immersive media. In addition to different strategies for different types of anxiety, we also look at possible automated measures and a current product implementation that already uses them.
Previous work on anxiety in virtual reality
Virtual reality has been used for some time in the therapeutic field, where patients are exposed to graded stress with anxiety triggers under professional supervision. Scientific studies have dealt with topics such as fear of flying or fear of heights (Wiederhold et al. 2002, Rotbaum et al. 1997). In 1995, Hodges et al. used a virtual floor in immersive VR applications to treat fear of heights.
Mods like “Spiders Begone” in Skyrim have replaced the virtual large spiders with bears in 2D computer games. In the VR game Walkabout Mini Golf VR, you have the option to disable the creepy large spiders in the attic of the golf course “Widow’s Walkabout” if you have a fear of spiders (arachnophobia).
Unlike these interactively rendered experiences, the images for immersive media, such as 3D photos, 180-degree videos, etc., are already finalized. Therefore, it is not possible to simply omit certain 3D objects in a rendering engine in order to re-render them or convert them to a different geometry. In this area of immersive media, there is a patent application from immerVR that summarizes 13 mitigation strategies and shows more specific algorithms in detail.
The following general points for reducing anxiety triggers in immersive images/videos can be found there:
- Removing the image/video from the play list
- Reduction of the visible area (field of view)
- Viewing the images through a small, movable lens
- Viewing the media on a small, fixed screen with options to rotate the displayed image
- Blurring the fear trigger
- Cut out the fear trigger and replace it with something else in a suitable context
- Display only in 2D instead of possible stereoscopic display
- Decrease the volume or eliminate sounds that induce anxiety.
- Remove haptic feedback
- Display a virtual floor under the user
- Change the assumed eye distance of the VR user to change the perceived size
- In 6-DoF media (3D media with free movement in space), position the user further away from the trigger
- Fade in elements of the real world
The basics
First, let’s look at the types of images we are dealing with in virtual reality. In general, much of what we describe here using individual images also applies to video. The most common image formats for virtual reality are
- Normal 2D photos (e.g. classic camera, cell phone)
- Short-angle 3D photos (similar to 3D cinema or 3D TV)
- VR180-3D media: a hemisphere is captured from two different views, usually at the distance of the average human eye distance of 6.3 cm
- 360-2D: a complete sphere of the environment, e.g. captured by 360° action cameras or drones
- 360-3D refers to an entire spherical environment that incorporates stereoscopy.
- Panoramas: very wide-angle panoramas, e.g. created by panning the smartphone
Image formats from left to right: 180° × 180° on a hemisphere, 360° × 180° on a full sphere, normal 2D image on a flat display surface and a panorama on a curved screen. | Image: alwaysVR
Certain image formats are not well-suited for standard 2D displays, hence certain widely-used algorithms, such as those for AI object recognition, may still require optimization for these formats.
The Application
A current version of the immersive VR image viewer immerGallery was used as the basis for the research. Some parts of the research have since been incorporated back into the product. In order to be able to react adequately to the individual fears of a user, it is possible to specify some of the user’s own fears in the application. In a broader sense, it would also be conceivable to have such a profile across applications, e.g. confidentially stored in the profile of the headset account.
Fears and solutions
After discussing various theoretical methods for addressing anxiety triggers, we will now focus on examining the practical application and the knowledge derived from it.
Image-based anxiety triggers
If you are aware of the specific elements in the image that may cause anxiety, you can attempt to eliminate them. During the bachelor thesis, we identified and highlighted the relevant components in each image by drawing a rectangle in a separate file. If the marked area is extensive and removing it would render the image unimportant for viewing, you can alternatively consider designating the entire image as the anxiety trigger, without using the rectangle.
As a simple option, we then filled the marked pixels of the rectangle with the surrounding color, which censors the scare trigger. Depending on the background, this can work very well or have a disturbing effect, depending on whether the borders of the rectangle are visible and the background has a very different color.
The most effective approach is to utilize AI methods that can fill in missing parts of an image using AI inpainting. It is preferable for the viewer to be unaware that the image has been modified. However, running these AI algorithms can be resource-intensive on existing standalone headsets. Therefore, as part of our research, we have only included the option of providing a pre-processed AI-altered image as an alternative solution for the specific anxiety trigger.
In this scenario, we consider the instance of a photograph featuring a beetle resting on a garden table. The individual has previously expressed their fear towards beetles or insects as a whole.
Left: Original image of a beetle on a table. Center: Gray rectangle of the same color over fear trigger. Blue border for clarification only. Right: Fear trigger was removed via AI inpainting. | Image: Source: immerVR
Immersive images have a unique aspect where one needs to eliminate anxiety-inducing elements on the left and right sides of a 3D photo. It’s important to ensure that the perceived depth of the image remains unaffected. When it comes to the rectangular shape, as long as it is relatively slim around the area being removed, it tends to work effectively in practical situations.
With AI inpainting, where pixels are randomly “invented” without the AI algorithms taking stereoscopy into account, false impressions of depth can occur more quickly. With VR180 and 360-degree images in equirectangular format, it should also be noted that the distortion of the format may need to be compensated for first, which will be discussed in more detail later.
Spatial-based anxiety triggers
In addition to image-based triggers, there are many triggers that are related to perceived space. For example, fear of heights can be triggered. Claustrophobia can be triggered by rooms that are too small. Being too close to other people or animals can trigger a fear of touch (haphepobia).
Creating a depth map is highly advantageous in evaluating various areas. Numerous tools are available for generating depth maps using both monoscopic and stereoscopic images. Nevertheless, converting the depth pixel values to real-world measurements is often ambiguous. However, if the precise specifications of a 3D camera, including lens properties, are known, it is possible to convert the measurements into metric units accurately. This enables a reasonable estimation of distances, such as the distance to the ground or other individuals.
In our research, we focused on the topic of vertigo. Studies have revealed that 3-6% of the population experiences severe anxiety related to this issue, and approximately 30% of individuals may feel some level of discomfort at high altitudes. Considering its significant prevalence, addressing anxiety in virtual reality related to vertigo becomes crucial, thankfully offering effective solutions.
As previously stated, previous research has proposed the use of a virtual floor to address anxiety in virtual reality. This approach has now been implemented and improved in the immerGallery product, specifically for users with acrophobia, through the inclusion of a user profile.
Different settings in the immerGallery user profile for fear of heights. | Image: immerGallery
For a moderate fear profile, an artificial floor with railings under the user’s feet is rendered as a 3D environment for all monoscopic images (e.g., drone photos or 360° action cam photos) and for all images displayed on a virtual screen. The high setting does this for all images.
Why do medium and high settings have a difference? The reason is that in Virtual Reality, the floor rendered artificially as a 3D object naturally has depth. When the content displayed is only monoscopic, there are no conflicts in depth perception. However, with stereoscopic media, the user sees images for each eye that are slightly offset, which tricks our perception into interpreting depth. This can clash with the depth of the virtual floor, leading to a more uncomfortable sense of depth. Nevertheless, if you have a strong fear of heights, this method may be preferable.
Left: The artificial floor designed to alleviate fear of heights does not cause a sense of depth conflict. Right: The railing appears to be closer compared to the floor obstructing it. | Image: alwaysVR
As the example in the figure shows, the artificial 3D floor against vertigo can also be rendered without perceptual problems when the elements are further apart, as in the left image. In the right (stereoscopic) image, there would be a depth conflict between the perception of the depth of the floor and the depth of the railing. Depending on the severity of the vertigo, the floor may be the better alternative despite the depth conflict.
Of course, an artificial object such as the 3D floor can also somewhat reduce the illusion of presence in virtual media when viewing highly immersive images. To avoid this as much as possible, we offer several other contextual floors to combat vertigo.
The basket of the hot air balloon provides support for fear of heights. | Image: Source: immerVR
Looking at snorkeling photos with a safe seabed under your feet against fear of heights. | Image: immerVR
As the two examples of the 3D hot air balloon and the 3D underwater environment show, it is certainly possible to provide contextualized floors to combat vertigo that have less impact on immersion.
Multisensory anxiety triggers
In an immersive VR slideshow, not only images are shown, but also sound and possibly haptics. For example, if a user has a fear of dogs, a barking soundtrack may trigger anxiety when a photo of a natural landscape without animals is shown. Therefore, as part of the bachelor thesis, we proposed to hide certain sequences of a marked audio track with anxiety triggers or to turn off an audio track with anxiety triggers completely.
In order to avoid unnatural gaps, alternative audio tracks could also be offered here, e.g. one where the dog sound is replaced by a cat sound and then loaded depending on the user’s anxiety profile. Haptics is often just the vibration of the controller when navigating through menus or during interactions such as switching images. But these haptic perceptions also increase the overall immersion.
If you are concerned about physical interaction in media that focuses on humans, you might want to think about disabling controller haptics. By using haptic VR gloves or haptic full-body suits, you will have the opportunity to explore more refined and innovative sensations.
Anxiety-inducing environments
ImmerGallery provides diverse virtual environments, including hot-air balloons and underwater settings, to enhance the immersion and dynamics. It also offers a meadow environment with pollen and butterflies for nature photography.
A user expressed their aversion towards butterflies and mentioned they would not utilize this setting. Consequently, we have developed a “Meadow without Butterflies” option, wherein the meadow environment remains lively with pollen activity but excludes butterflies, in order to avoid triggering fear.
Fear of the next image
The feedback from users has also made us aware that proceeding to the following image without the knowledge of its content can create uneasiness. It is technically feasible to access the menu and scroll to locate the thumbnail of the subsequent image. Nonetheless, in reality, this approach is excessively time-consuming and diminishes the sense of immersion. The straightforward resolution is to present a thumbnail of the next image on the controller.
A preview of the next image on the controller reduces the fear of unexpected content with possible anxiety triggers | Image: immerVR
Automate the detection and replacement of fear triggers
By manually tagging and replacing various anxiety triggers, it is already possible to create very pre-selected immersive content that can be consumed comfortably on a wide scale. Of course, we would like to automate this on a larger scale. We talked about this in more detail in the research paper and will discuss it briefly here.
The equirectangular format is commonly used to store hemispherical 180° × 180° and fully spherical 360° × 180° media. It can be visualized as converting a globe into a rectangular shape. However, this conversion results in noticeable distortions, as shown in the image.
Distortions in equirectangular format | Image: Tobias Jung
Most AI algorithms are primarily trained on typical, wide-angle images, which are the most common type of images. However, AI detectors often struggle to perform well on equirectangular images.
In the above image with the distortions, the text is not recognized by AI object recognition. It only works after rectification as in the image below. | Image: alwaysVR
To obtain satisfactory outcomes with current AI detectors, it is necessary to convert the spherical image format into alternative representations.
As mentioned above, AI-based inpainting for replacing image content usually does not train support for stereoscopic images, which automatically makes appropriate depth-corrected adjustments in the images for the left and right eyes. Unfortunately, a similar situation currently exists with AI-based photo upscaling.
The current tools are not equipped to generate consistent new details for the left and right eye images in 2D photos, thus limiting depth perception. Further research is necessary in this field, and we can anticipate the development of innovative products.
Summary
The issue of anxiety triggers in virtual reality will sooner or later confront many users as the medium becomes more widespread. While interactively rendered games can exchange 3D models with anxiety triggers for rendering, the situation is different for already captured 3D photos and 3D videos. In this article, we have already outlined many research approaches for reducing anxiety triggers in immersive media.
Some of these have already found their way into consumer products. However, much remains to be done in this area. Both on the research side and on the side of app developers and providers of AI tools to make VR even more accessible. It remains an important area with many exciting developments to come. We are pleased to have been able to report on this first phase and look forward to seeing where the journey takes us.
Daniel Pohl, the CEO and founder of immerVR GmbH, is actively involved in developing new ideas and advancements in immersive media, specifically in VR180 stereo photography. Using his app, immerGallery, users can enjoy incredibly immersive photo galleries on Meta Quest devices, complete with voice-overs, background music, and the option to engage in multiplayer experiences with friends.
