ChatGPT and me: First-time and experienced users’ perceptions of ChatGPT’s communicative ability as a dialogue partner

IF 5.3 2区计算机科学 Q1 COMPUTER SCIENCE, CYBERNETICS

International Journal of Human-Computer Studies Pub Date : 2024-11-04 DOI:10.1016/j.ijhcs.2024.103400

Iona Gessinger , Katie Seaborn , Madeleine Steeds , Benjamin R. Cowan

{"title":"ChatGPT and me: First-time and experienced users’ perceptions of ChatGPT’s communicative ability as a dialogue partner","authors":"Iona Gessinger , Katie Seaborn , Madeleine Steeds , Benjamin R. Cowan","doi":"10.1016/j.ijhcs.2024.103400","DOIUrl":null,"url":null,"abstract":"<div><div>Chatbots like ChatGPT have the potential to produce more natural conversational user interface interactions. Yet, we currently know little about perceptions of ChatGPT as a dialogue partner, and if interaction changes these. Through an online, two-stage, mixed methods study conducted in July 2023, in which first-time and experienced users living in the UK or Ireland engaged in tasks with ChatGPT, we show that interaction improves attitudes towards the system for first-time users, while these attitudes are already positive and stable in experienced users. We further show that first-time users’ perceptions of ChatGPT’s communicative ability (competence, human-likeness, and flexibility) are more dynamic than those of experienced users, although the experienced users’ perceptions also peak post-interaction. When reflecting on their interaction experience with ChatGPT, both groups were positive with little mention of limitations. We discuss the implications of these findings for user perceptions of ChatGPT as a dialogue partner, and highlight the potential risks of uncritical adoption of such technology.</div></div>","PeriodicalId":54955,"journal":{"name":"International Journal of Human-Computer Studies","volume":"194 ","pages":"Article 103400"},"PeriodicalIF":5.3000,"publicationDate":"2024-11-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"International Journal of Human-Computer Studies","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1071581924001836","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, CYBERNETICS","Score":null,"Total":0}

引用次数: 0

Abstract

Chatbots like ChatGPT have the potential to produce more natural conversational user interface interactions. Yet, we currently know little about perceptions of ChatGPT as a dialogue partner, and if interaction changes these. Through an online, two-stage, mixed methods study conducted in July 2023, in which first-time and experienced users living in the UK or Ireland engaged in tasks with ChatGPT, we show that interaction improves attitudes towards the system for first-time users, while these attitudes are already positive and stable in experienced users. We further show that first-time users’ perceptions of ChatGPT’s communicative ability (competence, human-likeness, and flexibility) are more dynamic than those of experienced users, although the experienced users’ perceptions also peak post-interaction. When reflecting on their interaction experience with ChatGPT, both groups were positive with little mention of limitations. We discuss the implications of these findings for user perceptions of ChatGPT as a dialogue partner, and highlight the potential risks of uncritical adoption of such technology.

查看原文本刊更多论文

我和 ChatGPT：首次用户和资深用户对 ChatGPT 作为对话伙伴的交流能力的看法

像 ChatGPT 这样的聊天机器人有可能产生更自然的对话式用户界面交互。然而，我们目前对 ChatGPT 作为对话伙伴的感知以及互动是否会改变这些感知知之甚少。通过 2023 年 7 月进行的一项在线、两阶段、混合方法研究，我们发现，交互改善了首次用户对系统的态度，而经验丰富的用户对系统的态度已经积极且稳定。我们进一步表明，与经验丰富的用户相比，初次使用 ChatGPT 的用户对其交流能力（能力、人性化和灵活性）的感知更为动态，尽管经验丰富的用户的感知也会在互动后达到顶峰。在反思与 ChatGPT 的交互体验时，两组用户都表现积极，很少提到局限性。我们讨论了这些发现对用户将 ChatGPT 视为对话伙伴的影响，并强调了不加批判地采用此类技术的潜在风险。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

International Journal of Human-Computer Studies 工程技术-计算机：控制论

CiteScore

11.50

自引率

5.60%

发文量

108

审稿时长

3 months

期刊介绍： The International Journal of Human-Computer Studies publishes original research over the whole spectrum of work relevant to the theory and practice of innovative interactive systems. The journal is inherently interdisciplinary, covering research in computing, artificial intelligence, psychology, linguistics, communication, design, engineering, and social organization, which is relevant to the design, analysis, evaluation and application of innovative interactive systems. Papers at the boundaries of these disciplines are especially welcome, as it is our view that interdisciplinary approaches are needed for producing theoretical insights in this complex area and for effective deployment of innovative technologies in concrete user communities. Research areas relevant to the journal include, but are not limited to: • Innovative interaction techniques • Multimodal interaction • Speech interaction • Graphic interaction • Natural language interaction • Interaction in mobile and embedded systems • Interface design and evaluation methodologies • Design and evaluation of innovative interactive systems • User interface prototyping and management systems • Ubiquitous computing • Wearable computers • Pervasive computing • Affective computing • Empirical studies of user behaviour • Empirical studies of programming and software engineering • Computer supported cooperative work • Computer mediated communication • Virtual reality • Mixed and augmented Reality • Intelligent user interfaces • Presence ...