Triadic human-agent dynamics in collaborative decision-making within extended reality
Type of content
Publisher's DOI/URI
Thesis discipline
Degree name
Publisher
Journal Title
Journal ISSN
Volume Title
Language
Date
Authors
Abstract
The convergence of Extended Reality (XR) and Artificial Intelligence (AI), particularly Large Language Models (LLMs), offers exciting opportunities to enhance collaborative decision-making processes. XR, encompassing virtual, augmented, and mixed reality, combined with LLMs, can power Embodied Virtual Agents (EVAs), virtual characters that communicate naturally with users in real time. This project investigates the role of LLM-driven EVAs in supporting group decision-making. We developed the XMAC Framework, a multiscale, multiuser VR system integrated with an LLM-powered embodied agent. This framework allows users to switch perspectives, interact with virtual objects, and communicate directly with the agent. A user study was conducted to investigate the effects of virtual agents on collaborative decision-making within an XR office design task. The study aimed to assess how the presence and embodiment of a virtual agent influence collaborative behaviours and teamwork efficiency, as well as how different user perspectives (Regular and Giant) affect interaction patterns and engagement with both embodied and disembodied agents. This study examined the effects of virtual agents, their embodiment, and user perspectives on collaborative decision-making tasks. The findings revealed that virtual agents influence key aspects of collaboration. The embodied agent enhanced the sense of co-presence, creating a more immersive experience for users. Additionally, participants communicated more frequently with the agent and spent more time in the regular perspective. Qualitative data highlighted varied user preferences, with some favouring the embodied agent’s engagement, while others preferred the disembodied agent’s less intrusive nature or no agent at all for greater autonomy. These results underscore the complexity of designing virtual agents in collaborative environments, as user preferences and working styles play a crucial role in their perceived effectiveness. The findings suggest that including a virtual agent influenced aspects of collaboration, such as perceived closeness and group effectiveness, but had no significant impact on task completion time or overall user satisfaction, suggesting that the agent shapes collaboration dynamics without necessarily improving task performance. Moreover, the embodied agent enhanced co-presence and made interactions more immersive, but it did not improve teamwork efficiency or interaction frequency, indicating that while embodiment boosts user engagement, it may not lead to better collaboration outcomes. Also, different perspectives affected user behaviour, with the Regular Perspective fostering more frequent interactions, longer engagement, and a stronger sense of co-presence. The Giant Perspective was useful for spatial awareness but less conducive to detailed interactions. Overall, virtual agents, virtual agent embodiment and user perspectives significantly influence collaboration and interaction in XR environments, with varied effects on user behaviour and engagement.