Enhancing supermarket robot interaction: an equitable multi-level LLM conversational interface for handling diverse customer intents.

IF 2.9 Q2 ROBOTICS
Frontiers in Robotics and AI Pub Date : 2025-04-29 eCollection Date: 2025-01-01 DOI:10.3389/frobt.2025.1576348
Chandran Nandkumar, Luka Peternel
{"title":"Enhancing supermarket robot interaction: an equitable multi-level LLM conversational interface for handling diverse customer intents.","authors":"Chandran Nandkumar, Luka Peternel","doi":"10.3389/frobt.2025.1576348","DOIUrl":null,"url":null,"abstract":"<p><p>This paper presents the design and evaluation of a comprehensive system to develop voice-based interfaces to support users in supermarkets. These interfaces enable shoppers to convey their needs through both generic and specific queries. Although customisable state-of-the-art systems like GPTs from OpenAI are easily accessible and adaptable, featuring low-code deployment with options for functional integration, they still face challenges such as increased response times and limitations in strategic control for tailored use cases and cost optimization. Motivated by the goal of crafting equitable and efficient conversational agents with a touch of personalisation, this study advances on two fronts: 1) a comparative analysis of four popular off-the-shelf speech recognition technologies to identify the most accurate model for different genders (male/female) and languages (English/Dutch) and 2) the development and evaluation of a novel multi-LLM supermarket chatbot framework, comparing its performance with a specialized GPT model powered by the GPT-4 Turbo, using the Artificial Social Agent Questionnaire (ASAQ) and qualitative participant feedback. Our findings reveal that OpenAI's Whisper leads in speech recognition accuracy between genders and languages and that our proposed multi-LLM chatbot architecture, which outperformed the benchmarked GPT model in performance, user satisfaction, user-agent partnership, and self-image enhancement, achieved statistical significance in these four key areas out of the 13 evaluated aspects that all showed improvements. The paper concludes with a simple method for supermarket robot navigation by mapping the final chatbot response to the correct shelf numbers to which the robot can plan sequential visits. Later, this enables the effective use of low-level perception, motion planning, and control capabilities for product retrieval and collection. We hope that this work encourages more efforts to use multiple specialized smaller models instead of always relying on a single powerful model.</p>","PeriodicalId":47597,"journal":{"name":"Frontiers in Robotics and AI","volume":"12 ","pages":"1576348"},"PeriodicalIF":2.9000,"publicationDate":"2025-04-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12069059/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Frontiers in Robotics and AI","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.3389/frobt.2025.1576348","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2025/1/1 0:00:00","PubModel":"eCollection","JCR":"Q2","JCRName":"ROBOTICS","Score":null,"Total":0}
引用次数: 0

Abstract

This paper presents the design and evaluation of a comprehensive system to develop voice-based interfaces to support users in supermarkets. These interfaces enable shoppers to convey their needs through both generic and specific queries. Although customisable state-of-the-art systems like GPTs from OpenAI are easily accessible and adaptable, featuring low-code deployment with options for functional integration, they still face challenges such as increased response times and limitations in strategic control for tailored use cases and cost optimization. Motivated by the goal of crafting equitable and efficient conversational agents with a touch of personalisation, this study advances on two fronts: 1) a comparative analysis of four popular off-the-shelf speech recognition technologies to identify the most accurate model for different genders (male/female) and languages (English/Dutch) and 2) the development and evaluation of a novel multi-LLM supermarket chatbot framework, comparing its performance with a specialized GPT model powered by the GPT-4 Turbo, using the Artificial Social Agent Questionnaire (ASAQ) and qualitative participant feedback. Our findings reveal that OpenAI's Whisper leads in speech recognition accuracy between genders and languages and that our proposed multi-LLM chatbot architecture, which outperformed the benchmarked GPT model in performance, user satisfaction, user-agent partnership, and self-image enhancement, achieved statistical significance in these four key areas out of the 13 evaluated aspects that all showed improvements. The paper concludes with a simple method for supermarket robot navigation by mapping the final chatbot response to the correct shelf numbers to which the robot can plan sequential visits. Later, this enables the effective use of low-level perception, motion planning, and control capabilities for product retrieval and collection. We hope that this work encourages more efforts to use multiple specialized smaller models instead of always relying on a single powerful model.

增强超市机器人交互:一个公平的多层次LLM会话接口,用于处理不同的客户意图。
本文介绍了一个综合系统的设计和评估,以开发基于语音的界面,以支持超市用户。这些接口使购物者能够通过通用查询和特定查询来传达他们的需求。尽管像OpenAI的gpt这样的可定制的最先进的系统易于访问和适应,具有低代码部署和功能集成选项,但它们仍然面临着诸如响应时间增加和定制用例和成本优化的战略控制限制等挑战。本研究的目标是创造公平、高效的会话代理,并带有一点个性化,因此本研究在两个方面取得了进展:1)比较分析了四种流行的现有语音识别技术,以确定不同性别(男性/女性)和语言(英语/荷兰语)的最准确模型;2)开发和评估了一种新的多llm超市聊天机器人框架,使用人工社会代理问卷(ASAQ)和定性参与者反馈,将其性能与由GPT-4 Turbo驱动的专用GPT模型进行了比较。我们的研究结果表明,OpenAI的Whisper在性别和语言之间的语音识别准确性方面处于领先地位,我们提出的多llm聊天机器人架构在性能、用户满意度、用户代理合作伙伴关系和自我形象增强方面优于基准GPT模型,在这四个关键领域取得了统计显著性,13个评估方面都显示出了改进。本文最后提出了一种简单的超市机器人导航方法,通过将最终的聊天机器人响应映射到正确的货架编号,机器人可以计划顺序访问。之后,这使得能够有效地使用低级感知、运动规划和控制功能来进行产品检索和收集。我们希望这项工作鼓励更多的努力,使用多个专门的较小的模型,而不是总是依赖于一个强大的模型。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
CiteScore
6.50
自引率
5.90%
发文量
355
审稿时长
14 weeks
期刊介绍: Frontiers in Robotics and AI publishes rigorously peer-reviewed research covering all theory and applications of robotics, technology, and artificial intelligence, from biomedical to space robotics.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信