Beyond Reality: Designing Personal Experiences and Interactive Narratives in AR Theater

You-Jin Kim:
Beyond Reality: Designing Personal Experiences and Interactive Narratives in AR Theater. In: University of California, Santa Barbara, Dissertation 2024
Presented: UCSB - November 8, 2024, Santa Barbara, California

▪️  ResearchGate: researchgate/396967048    ▪️  arXiv: 10.48550/arXiv.2510.22098

Abstract

Augmented Reality (AR) technologies are redefining how we perceive and interact with the world by seamlessly integrating digital elements into our physical surroundings. These technologies offer personalized experiences and transform familiar spaces by layering new narratives onto the real world.

Through increased levels of perceived agency and immersive environments, my work aims to merge the human elements of live theater with the dynamic potential of virtual entities and AI agents. This approach captures the subtlety and magic of storytelling, making theater experiences available anytime and anywhere. The system I am building introduces innovative methods for theatrical production in virtual settings, informed by my research and eight published works. These contributions highlight domain-specific insights that have shaped the design of an immersive AR Theater system.

My research in building a well-designed AR stage features avatars and interactive elements that allow users to engage with stories at their own pace, granting them full agency over their experience. However, to ensure a smooth and curated experience that aligns with the director or creator's vision, several factors must be considered, especially in open-world settings that depend on natural user movement. This requires the story to be conveyed in a controlled manner, while the interaction remains intuitive and natural for the user.

Research Contributions

• Immersive AR Theater Framework: Developed an entertainment platform supporting interactive narratives, natrual locomotion, and spatially defined content zones.

• Wide-area Outdoor AR Studies: Conducted an outdoor user study evaluating how wide-area AR navigation cues impact spatial memory and object recall.

• Embodied AI-Agent Pipeline: Developed an avatar-based DNN trained via reinforcement learning in digital twins to adapt to physical layouts.

• Gesture and Behavior Learning: Transferred learned human behaviors and gestures from task-specific tracking data to AI avatars.

• Future Content: Immersive, spatially-anchored content will place volumetric virtual elements into blended environments at specific physical locations.

Citation IEEE Format

[1] Y-J. Kim, "Beyond Reality: Designing Personal Experiences and Interactive Narratives in AR Theater," Ph.D. dissertation, Univ. California, Santa Barbara, CA, USA, 2024, Order No. 31761773. [Online]. Available: ProQuest Dissertations, doi: 10.48550/arXiv.2510.22098. (189 Pages)

Citation APA Format

Kim, Y-J. (2024). Beyond Reality: Designing Personal Experiences and Interactive Narratives in AR Theater (Publication No. 31761773) [Doctoral dissertation, University of California, Santa Barbara]. ProQuest Dissertations. https://doi.org/10.48550/arXiv.2510.22098. (189 Pages)