Investigating Search Among Physical and Virtual Objects Under Different Lighting Conditions

You-Jin Kim, Radha Kumaran, Ehsan Sayyad, A Milner, Tom Bullock, Barry Giesbrecht, and Tobias Höllerer:
Investigating Search Among Physical and Virtual Objects Under Different Lighting Conditions. In: IEEE Transactions on Visualization and Computer Graphics, IEEE TVCG 2022
Presented: IEEE ISMAR 2022 - [Evaluation Session] October 19, 2022, Singapore

▪️  DOI: 10.1109/TVCG.2022.3203093    ▪️  arXiv: 10.48550/arXiv.2511.00289

Abstract

By situating computer-generated content in the physical world, mobile augmented reality (AR) can support many tasks that involve effective search and inspection of physical environments. Currently, there is limited information regarding the viability of using AR in realistic wide-area outdoor environments and how AR experiences affect human behavior in these environments. Here, we conducted a wide-area outdoor AR user study (n=48) using a commercially available AR headset (Microsoft Hololens 2) to compare (1) user interactions with physical and virtual objects in the environment (2) the effects of different lighting conditions on user behavior and AR experience and (3) the impact of varying cognitive load on AR task performance. Participants engaged in a treasure hunt task where they searched for and classified virtual target items (green “gems”) in an augmented outdoor courtyard scene populated with physical and virtual objects. Cognitive load was manipulated so that in half the search trials users were required to monitor an audio stream and respond to specific target sounds. Walking paths, head orientation and eye gaze information were measured, and users were queried about their memory of encountered objects and provided feedback on the experience. Key findings included (1) Participants self-reported significantly lower comfort in the ambient natural light condition, with virtual objects more visible and participants more likely to walk into physical objects at night; (2) recall for physical objects was worse than for virtual objects, (3) participants discovered more gems hidden behind virtual objects than physical objects, implying higher attention on virtual objects and (4) dual-tasking modified search behavior. These results suggest there are important technical, perceptual and cognitive factors that must be considered if the full potential of “anywhere and anytime mobile AR” is to be realized.

Research Contributions

• Outdoor AR search task completion time and distance traveled significantly decrease over sequential trials, demonstrating strong spatial learning effects.

• Head rotation increases across trials as users learn the environment, but decreases under dual-task conditions due to cognitive load.

• Free-floating virtual gems are localized more accurately than target objects hidden behind either physical or virtual occlusions.

• Target discrimination is significantly more accurate for virtually occluded objects than physically occluded objects, suggesting an AR visibility bias.

• Immersive AR experiences impair the recall of purely physical objects relative to virtual or multi-modal counterparts in the environment.

Citation IEEE Format

[1] Y-J. Kim et al., "Investigating Search Among Physical and Virtual Objects Under Different Lighting Conditions," IEEE Trans. Vis. Comput. Graph., vol. 28, no. 11, pp. 3788-3798, Nov. 2022, doi: 10.1109/TVCG.2022.3203093. (11 Pages)

Citation APA Format

Kim, Y-J., Kumaran, R., Sayyad, E., Milner, A., Bullock, T., Giesbrecht, B., & Höllerer, T. (2022). Investigating search among physical and virtual objects under different lighting conditions. IEEE Transactions on Visualization and Computer Graphics, 28(11), 3788–3798. https://doi.org/10.1109/TVCG.2022.3203093. (11 Pages)