Does Embodiment Still Matter? Comparing User Experience with LLM-Powered Agents

Conversational Artificial Intelligence (AI) is increasingly integrated into various aspects of human life, yet creating truly natural and engaging interactions remains a challenge. The role of physical embodiment in shaping user experience, particularly when coupled with advanced AI capabilities, requires further investigation. This study aimed to investigate the impact of embodiment on user experience by comparing interactions with an embodied conversational agent (Pepper robot) versus a non-embodied agent (Laptop AI), both powered by the same sophisticated Large Language Model (LLM), Ollama Llama 3.2 7B. A within-subjects experiment (N = 32) was conducted where participants interacted with both the embodied and non-embodied agents in a counterbalanced order. The agents utilised the Ollama Llama 3.2 7B model and Google Text-to-Speech. Post-interaction questionnaires assessed user experience on 5-point Likert scales. Results revealed significantly higher user ratings for the embodied Pepper robot across multiple dimensions, including perceived naturalness (p =.001), conversation flow (p =.010), understanding of agent responses (p =.005), relevance of responses (p =.027), user engagement (p <.001), perception as a social entity (p =.002), sense of connection (p <.001), and comfort (p =.024). Participants expressed a unanimous and strong preference for the embodied agent. Crucially, self-rated tech-savviness did not significantly correlate with these core interaction metrics for either agent type. Furthermore, the embodied agent met user expectations for naturalness, whereas the non-embodied agent did not (p =.002). Findings demonstrate that physical embodiment, when combined with an advanced LLM, substantially enhances user experience, fostering more natural, engaging, and socially resonant interactions compared to an equivalent non-embodied system, largely independent of user tech-savviness.