Current Projects (13)
ML2 + Emotiv Epoc X - 14 Channel Wireless EEG Headset, AR Physical Cockpit, Flight Simulation, Spatial Interaction, ACM CHI 2027
AR headsets to extend the physical cockpit, examine cognitive load during attentional transitions between the cockpit, virtual interfaces and flight view.
[details redacted for review] First Author: Thomas Marxsen
In Development
EEG-AR-AugCog-Cockpit
ACM CHI 2027
Full Paper
EEG + ML2
Key words: Augmented Reality, Outdoor Wide-area, Spatial Awareness, Memory, Cognitive Load, Augmented cognition, IEEE VR 2027
Investigates how highlighted physical grid overlays enhance day and night outdoor virtual object localization during map-based search tasks.
[details redacted for review] First Author: Jared V. Samonte
In Development
DRL-AR-SpatialOutdoor
IEEE VR 2027
Full Paper
Magic Leap 2
Key words: Cultural Heritage, Virtual Reality, Digital Twin Archives, Historical Site, Preservation, Interactive Narrative, AI-Agent, ACM CHI 2027
Experiential reminiscence and heritage education through interactive narrative through digital twins of displaced communities in Gering, Nebraska
[details redacted for review] First Author: Lijun Mao
In Development
EDU-VR-MesumeAgent
ACM CHI 2027
Full Paper
Meta Quest Pro
Key words: AR Software Development, Interdisciplinary Collaboration, Project-based Learning, HCI Education, Pedagogy, ACM CHI 2027
Study investigates Interdisciplinary project-based course enhances technical proficiency and creative problem-solving in AR software development.
[details redacted for review] First Author: You-Jin Kim
Ready for Submission
EDU-AR-CollegeCourse
ACM CHI 2027
Full Paper
HoloLens 2
Key words: Co-located Interaction, Natrual Collaborative Interaction, Co-presence, Avatar Emobiemnt, Social Dynamics, ACM CHI 2027
Projection-based augmentation fosters superior physical coordination and communication in multi-user SAR settings.
[details redacted for review] First Author: You-Jin Kim
Ready for Submission
SAR-AR-VR-SharedSpace
ACM CHI 2027
Full Paper
Spatial AR
Key words: Co-located Interaction, Group Locomotion, Multi-user VR, Spatial Awareness, Shared Experience, IEEE VR 2027
Shared-centroid locomotion transforms group movement into a social negotiation, balancing individual agency with coordination.
[details redacted for review] First Author: You-Jin Kim
Ready for Submission
SAR-SharedSpace-Locomotion
IEEE VR 2027
Full Paper
Spatial AR
Key words: Sign Language, Avatars Embodiment, Head Motion, Affective Computing, Accessibility, Embodied Communication, ACM CHI 2027
Study investigates framework for expressive head motion significantly improves social realism and engagement in signing avatars.
[details redacted for review] First Author: You-Jin Kim
Ready for Submission
DNN-AI-ASL-Avatar
ACM CHI 2027
Full Paper
Spatial AR
Key words: Interactive Music Systems, Kinesthetic Interaction, Full-body Tracking, Digital Synthesis, Performing Arts, ACM VRST 2026
Large-scale projection environment transforms physical space into a gestural musical instrument for expressive, full-body performance.
[details redacted for review] First Author: You-Jin Kim
Ready for Submission
SAR-Dance-SubStrata
ACM VRST 2026
Full Paper
Spatial AR
Key words: Navigation Aids, Spatial Search, Cognitive Load, Visual Cue, Attention, User Strategy, Adaptable Interfaces, ACM VRST 2026
Study investigates user strategies and attention shifts when using multiple navigation aids in AR.
[details redacted for review] First Author: Radha Kumaran
Ready for Submission
NSF-Wide-area-AR-4
ACM VRST 2026
Full Paper
HoloLens 2
Key words: Text Input, Finger-joint Mapping, Passive Haptic, Eyes-free Entry, Socially Discreet Input, ACM SUI 2026
Study presents finger-joint mapping system in AR enables discreet, haptic-based text entry for eyes-free augmented reality use.
[details redacted for review] First Author: Bid Pokhrel
Under Review
SUI-AR-JointTyping
ACM SUI 2026
Full Paper
Meta Quest 3
Key words: Object Trajectory, Extended Reality, Cognitive Attention, Memory, EEG, Eye-tracking Project, Nature Communications
Study validates framework for decoding spatial attention using oscillatory brain activity and eye-tracking.
[details redacted for review] First Author: Tom Bullock
Under Review
EEG-XR-SABER-3
Nature Communications
Journal Paper
Vive Pro Eyes 2
Key words: Spatial Uncertainty, Extended Reality, Cognitive Attention, Memory, Oscillatory Project, Community Biology
Neural modeling reconstructs moment-by-moment attention to dynamic oscillatory objects within immersive virtual environments.
[details redacted for review] First Author: Tom Bullock
Under Review
EEG-XR-SABER-4
Community Biology
Journal Paper
Vive Pro Eyes 2
Key words: Cognitive Load, Object Recall, Attention, Semantic Congruency, Visual Saliency, Spatial Cognition, IEEE ISMAR 2026
Study investigates how virtuality, saliency, and cognitive load influence object recall in mixed-reality environments.
[details redacted for review] First Author: Sangita Kunapuli
Accepted
NSF-AR-VisualSaliency
IEEE ISMAR 2026
Full Paper
Magic Leap 2
Presented Projects (18)
Radha Kumaran, You-Jin Kim, Shane Dirksen, Tom Bullock, Barry Giesbrecht, and Tobias Höllerer:
Challenges and Opportunities for Head-Worn AR Use in Wide-Area Environments. In: ACM Conference on Human Factors in Computing Systems, ACM CHI 2026 Workshop
Presented: ACM CHI 2026 - [Workshop 42: Next Steps for Augmented Reality On-the-Move] April 16, 2026 Barcelona, Spain
Tom Bullock, Emily Machniak, You-Jin Kim, Radha Kumaran, Justin Kasowski, Apurv Varshney, Julia Ram, Melissa M. Hernandez, Stina Johansson, Neil M. Dundon, Tobias Höllerer, Barry Giesbrecht:
SABER: Spatial Attention, Brain, Extended Reality. In: IEEE Conference on Virtual Reality and 3D User Interfaces, IEEE VR 2026
🏆 MoBI Award 2026 [Top Ranked Publications] Presented: IEEE VR 2026 - [Brain Sensing and Stimulation Session] March 25, 2026, Daegu, Korea
▪️ DOI: 10.1109/VR67842.2026.00042 ▪️ arXiv: 10.48550/arXiv.2603.24830
Shane Dirksen, Radha Kumaran, You-Jin Kim, Yilin Wang, Tobias Höllerer:
Modeling Object Attention in Mobile AR for Intrinsic Cognitive Security. In: ACM International Symposium on Theory, Algorithmic Foundations, and Protocol Design for Mobile Networks and Mobile Computing, ACM MobiHoc 2025
🏆 Presented: ACM MobiHoc 2025 - [Spatial Awareness Sessions] October 23, 2025, Houston, Texas
▪️ DOI: 10.1145/3704413.3765308 ▪️ arXiv: 10.48550/arXiv.2511.16805
Radha Kumaran, You-Jin Kim, Emily Machniak, Shane Dirksen, Junhyung Yoon, Tom Bullock, Barry Giesbrecht, Tobias Höllerer:
Scene Awareness While Using Multiple Navigation Aids in AR Search. In: IEEE International Symposium on Mixed and Augmented Reality, IEEE ISMAR 2025 Poster
Presented: IEEE ISMAR 2025 - [Poster Sessions] October 9, 2025, Daejeon, Korea
▪️ DOI: 10.1109/ISMAR-Adjunct68609.2025.00135 ▪️ arXiv: 10.48550/arXiv.2511.16805
Tom Bullock, Emily Machniak, Neil Dundon, Justin Kasowski, You-Jin Kim, Radha Kumaran, Julia Ram, Melissa Hernandez, Stina Johansson, Tobias Höllerer, Barry Giesbrecht:
Reconstructing moving object locations in an immersive 3D virtual environment from EEG oscillatory activity. In: The Association for Research in Vision and Ophthalmology, Journal of Vision 2025
▪️ DOI: 10.1167/jov.25.9.2876 ▪️ ResearchGate: researchgate.net/393741325
Tom Bullock, Emily Machniak, Joyce Passananti, Kangyou Yu, Radha Kumaran, You-Jin Kim, Jordan Garrett, Madhav Viswesvaran, Julia Ram, Tobias Höllerer, Barry Giesbrecht:
Electrophysiological evidence for differential semantic processing of words and objects presented in augmented reality. In: Elsevier, Computers in Human Behavior Reports, Elsevier CHBR 2025
▪️ DOI: 10.1016/j.chbr.2025.100677 ▪️ ResearchGate: researchgate.net/391364053
You-Jin Kim, Radha Kumaran, Jingjing Luo, Tom Bullock, Barry Giesbrecht, and Tobias Höllerer:
On the Go with AR: Attention to Virtual and Physical Targets while Varying Augmentation Density. In: ACM Conference on Human Factors in Computing Systems, ACM CHI 2025
Presented: ACM CHI 2025 - [Virtual and Mixed Reality Interaction Session] April 30, 2025, Yokohama, Japan
▪️ DOI: 10.1145/3706598.3714289 ▪️ arXiv: 10.48550/arXiv.2510.25978
Jamie Ngoc Dinh, Snehesh Shrestha, You-Jin Kim, Jun Nishida, Myungin Lee:
NeuResonance: Exploring Feedback Experiences for Fostering the Inter-brain Synchronization. In: ACM Conference on Human Factors in Computing Systems, ACM CHI 2025
Presented: ACM CHI 2025 - [Embodied Simulation Session] May 1, 2025, Yokohama, Japan
▪️ DOI: 10.1145/3706598.3713872 ▪️ arXiv: 10.48550/arXiv.2511.02079
Jamie Ngoc Dinh, Snehesh Shrestha, You-Jin Kim, Jun Nishida, Myungin Lee:
Demonstrating NeuResonance: Inter-brain Synchronization and Feedback Modalities in Collaboration. In: ACM Conference on Human Factors in Computing Systems, ACM CHI 2025 Interactivity
Presented: ACM CHI 2025 - [Interactivity Sessions] April 26–May 1, 2025, Yokohama, Japan
▪️ DOI: 10.1145/3706599.3721276 ▪️ ResearchGate: researchgate.net/391152470
You-Jin Kim:
Beyond Reality: Designing Personal Experiences and Interactive Narratives in AR Theater. In: University of California, Santa Barbara, Dissertation 2024
Presented: UCSB - November 8, 2024, Santa Barbara, California
▪️ ResearchGate: researchgate/396967048 ▪️ arXiv: 10.48550/arXiv.2510.22098
You-Jin Kim, Misha Sra, and Tobias Höllerer:
Audience Amplified: Virtual Audiences in Asynchronously Performed AR Theater. In: IEEE International Symposium on Mixed and Augmented Reality, IEEE ISMAR 2024
Presented: IEEE ISMAR 2024 - [Avatar 2 Session] October 24, 2024, Bellevue, Greater Seattle area
▪️ DOI: 10.1145/3706598.3713872 ▪️ arXiv: 10.48550/arXiv.2511.02807
Jamie Ngoc Dinh, You-Jin Kim, Myungin Lee:
FractalBrain: A Neuro-interactive Virtual Reality Experience using Electroencephalogram (EEG) for Mindfulness. In: ACM Conference on Human Factors in Computing Systems, ACM CHI 2024 Interactivity
Presented: ACM CHI 2024 - [Interactivity Sessions] May 11–16, 2024, Honolulu, Hawaiʻi
▪️ DOI: 10.1145/3613905.3648667 ▪️ arXiv: 10.48550/arXiv.2510.26041
You-Jin Kim, Myungin Lee, JoAnn Kuchera-Morin, and Tobias Höllerer:
Spatial Orchestra: Locomotion Music Instruments through Spatial Exploration. In: ACM Conference on Human Factors in Computing Systems, ACM CHI 2024 Interactivity
Presented: ACM CHI 2024 - [Interactivity Sessions] May 11–16, 2024, Honolulu, Hawaiʻi
▪️ DOI: 10.1145/3613905.3648659 ▪️ arXiv: 10.48550/arXiv.2510.23848
You-Jin Kim, Andrew D Wilson, Jennifer Jacobs, and Tobias Höllerer:
Reality Distortion Room: A Study of User Locomotion Responses to Spatial Augmented Reality Effects. In: IEEE International Symposium on Mixed and Augmented Reality, IEEE ISMAR 2023
Presented: IEEE ISMAR 2023 - [Navigation and Locomotion Session] October 18, 2023, Sydney, Australia
▪️ DOI: 10.1109/ISMAR59233.2023.00137 ▪️ arXiv: 10.48550/arXiv.2510.23840
You-Jin Kim, Joshua Lu, and Tobias Höllerer:
Dynamic Theater: Location-Based Immersive Dance Theater, Investigating User Guidance and Experience. In: ACM Symposium on Virtual Reality Software and Technology, ACM VRST 2023
Presented: ACM VRST 2023 - [Technologies in the Wild Session] October 10, 2023, Christchurch, New Zealand
▪️ DOI: 10.1145/3611659.3615705 ▪️ arXiv: 10.48550/arXiv.2511.00961
Radha Kumaran, You-Jin Kim, A Milner, Tom Bullock, Barry Giesbrecht, and Tobias Höllerer:
The Impact of Navigation Aids on Search Performance and Object Recall in Wide-Area Augmented Reality. In: ACM Conference on Human Factors in Computing Systems, ACM CHI 2023
Presented: ACM CHI 2023 - [Supporting users in AR and VR Session] April 26, 2023, Hamburg, Germany
▪️ DOI: 10.1145/3544548.3581413 ▪️ arXiv: 10.48550/arXiv.2510.25957
You-Jin Kim, Radha Kumaran, Ehsan Sayyad, A Milner, Tom Bullock, Barry Giesbrecht, and Tobias Höllerer:
Investigating Search Among Physical and Virtual Objects Under Different Lighting Conditions. In: IEEE Transactions on Visualization and Computer Graphics, IEEE TVCG 2022
Presented: IEEE ISMAR 2022 - [Evaluation Session] October 19, 2022, Singapore
▪️ DOI: 10.1109/TVCG.2022.3203093 ▪️ arXiv: 10.48550/arXiv.2511.00289
Zhaozhong Wang and You-Jin Kim:
Exploring Immersive Mixed Reality Simulations and Its Impact on Climate Change Awareness. In: Asian Journal of Applied Science and Engineering, Volume 12, ISSN: 2307-9584, AJASE 2022
Presented: ACM SIGGRAPH SPARKS 2023 - [Spatiality in Virtual Reality Session] January 27, 2023, Remote Presentation
▪️ DOI: 10.18034/AJASE.v11i1.3 ▪️ ResearchGate: researchgate/370597995
Video Presentation (7)
Spatial Orchestra
Spatial Orchestra: Locomotion Music Instruments through Spatial Exploration
ACM CHI 2024, Honolulu, Hawaiʻi
FractalBrain
FractalBrain: A Neuro-interactive Virtual Reality Experience using Electroencephalogram (EEG) for Mindfulness
ACM CHI 2024, Honolulu, Hawaiʻi
Reality Distortion Room
Reality Distortion Room: A Study of User Locomotion Responses to Spatial Augmented Reality Effects
IEEE ISMAR 2023, Sydney, Australia
Dynamic Theater
Dynamic Theater: Location-Based Immersive Dance Theater, Investigating User Guidance and Experience
ACM VRST 2023, Christchurch, New Zealand
Navigation Aids in Wide-Area AR
The Impact of Navigation Aids on Search Performance and Object Recall in Wide-Area Augmented Reality
ACM CHI 2023, Hamburg, Germany
Lighting Conditions in Wide-Area AR
Investigating Search Among Physical and Virtual Objects Under Different Lighting Conditions
IEEE ISMAR 2022, Singapore
World Tracer
World Tracer: An Easy-to-use Program that Creates the Layout of 3D Environments from LiDAR SLAM Data
Nokia Bell Labs Intern Day 2022, Murray Hill, New Jersey, USA
Open-source Projects (6)
CrowdPlayer
Kim, You-Jin (2024). CrowdPlayer (Version 1.0.2) [Computer software] Repository.
Unity template for 360° spatial AR featuring multi-camera stereoscopic body tracking.
▪️ DOI: 10.5281/zenodo.6565136 ▪️ GitHub: github.com/yujnkm/CrowdPlayer
JourneyReplay
Kim, You-Jin (2024). JourneyReplay (Version 1.0.2) [Computer software] Repository.
Replays HMD-tracked outdoor AR tracking data via digital twins to analyze user behavior and perspective.
▪️ DOI: 10.5281/zenodo.6565136 ▪️ GitHub: github.com/yujnkm/JourneyReplay
WorldTracer
Kim, You-Jin & Sharkey, Tommy (2023). WorldTracer (Version 1.0.0) [Computer software] Repository.
Reconstructing 3D factory floor models from live robotic SLAM data for DNN agent training.
▪️ DOI: 10.5281/zenodo.6565136 ▪️ GitHub: github.com/yujnkm/WorldTracer
3D Virtual Campus Project
Kim, You-Jin (2022). UCSB_3D Campus (Version 1.0.1) [Computer software] Repository.
A digital twin of the UC Santa Barbara campus, featuring 3D reconstructed models of KITP.
▪️ DOI: 10.5281/zenodo.6565136 ▪️ GitHub: github.com/yujnkm/VirtualUCSB
Immersive Storytelling Toolkit
Kim, You-Jin (2021). UCSB_3D Campus (Version 1.0.1) [Computer software] Repository.
A digital twin of the Santa Barbara campus for research and simulation, featuring three campus locations.
▪️ DOI: 10.5281/zenodo.6565136 ▪️ GitHub: github.com/yujnkm/UCSB_3D