Journey Replay: User Experience Playback Software for Wide-area Outdoor Augmented Reality

JourneyReplay
Kim, You-Jin (2024). JourneyReplay (Version 1.0.2) [Computer software] Repository.
Replays HMD-tracked outdoor AR  tracking data via digital twins to analyze user behavior and perspective.
▪️  DOI: 10.5281/zenodo.6565136    ▪️  GitHub: github.com/yujnkm/JourneyReplay

Abstract

Evaluating spatial user experience and behavioral reasoning wide-area outdoor augmented reality (AR) requires the precise synchronization of spatial, physiological, and interaction data. This open-source playback and analysis framework designed to reconstruct AR sessions within a high-fidelity digital twin of the physical environment. By aggregating telemetry from head-mounted displays (HMDs), the system re-renders the user’s percise field-of-view (FOV) perception, incorporating six-degree-of-freedom head movements and eye-gaze vectors to visualize visual attention relative to the environment. Eye gaze show as red ball moving around, where user is watching at tany moment.The interface utilizes a dual-pane visualization strategy: the left segment displays the egocentric FOV reconstruction, while the right segment provides an interactive 3D model of the digital twin, allowing for dynamic camera adjustments and external observation of user-spatial relationships. JourneyReplay integrates a geospatial map that tracks user trajectories alongside performance metrics, such as task completion efficiency and the accuracy of physical and virtual object identification. The software supports advanced multimodal analysis by visualizing synchronized electroencephalography (EEG) signals, enabling researchers to correlate cognitive states with specific environmental triggers or manual inputs, such as external controller status such as button presses. A temporal navigation bar at the base of the interface facilitates non-linear playback, allowing for the granular review of session progress and remaining duration. 

Research Contributions

• The playback software syncs movement to the map, allowing viewers to see how environmental situations influence user behavior and decision-making.

• The framework captures how users navigate while identifying specific patterns of movement, locomotion, and navigation, even showing input device results.

• The software enables users to replay and analyze behavior post-session, facilitating automated evaluation of behavioral insights discovered after the trial concludes.

• Integrating wide-area tracking with eye-tracking provides clear insights into what users focus on and how those visuals affect their navigation.

• FOV playback accurately representing the user’s perspective using digital twins helps measure cognitive load and spatial awareness across outdoor AR environments.

Citation IEEE Format

[1] Y-J. Kim, "Journey Replay: User Experience Playback Software for Wide-area Outdoor Augmented Reality," version 1.0.2, Zenodo, 2024. [Online]. Available: https://doi.org/10.5281/zenodo.6565136. GitHub: https://github.com/yujnkm/JourneyReplay

Citation APA Format

Kim, Y-J. (2024). UCSB_3D Campus: Journey Replay: User Experience Playback Software for Wide-area Outdoor Augmented Reality (Version 1.0.2) [Computer software]. Zenodo. https://doi.org/10.5281/zenodo.6565136; GitHub: https://github.com/yujnkm/JourneyReplay