From Data Stories to Dialogues: A Randomised Controlled Trial of Generative AI Agents and Data Storytelling in Enhancing Data Visualisation Comprehension

arXiv - CS - Human-Computer Interaction Pub Date : 2024-09-18 DOI:arxiv-2409.11645

Lixiang Yan, Roberto Martinez-Maldonado, Yueqiao Jin, Vanessa Echeverria, Mikaela Milesi, Jie Fan, Linxuan Zhao, Riordan Alfredo, Xinyu Li, Dragan Gašević

{"title":"From Data Stories to Dialogues: A Randomised Controlled Trial of Generative AI Agents and Data Storytelling in Enhancing Data Visualisation Comprehension","authors":"Lixiang Yan, Roberto Martinez-Maldonado, Yueqiao Jin, Vanessa Echeverria, Mikaela Milesi, Jie Fan, Linxuan Zhao, Riordan Alfredo, Xinyu Li, Dragan Gašević","doi":"arxiv-2409.11645","DOIUrl":null,"url":null,"abstract":"Generative AI (GenAI) agents offer a potentially scalable approach to support\ncomprehending complex data visualisations, a skill many individuals struggle\nwith. While data storytelling has proven effective, there is little evidence\nregarding the comparative effectiveness of GenAI agents. To address this gap,\nwe conducted a randomised controlled study with 141 participants to compare the\neffectiveness and efficiency of data dialogues facilitated by both passive\n(which simply answer participants' questions about visualisations) and\nproactive (infused with scaffolding questions to guide participants through\nvisualisations) GenAI agents against data storytelling in enhancing their\ncomprehension of data visualisations. Comprehension was measured before,\nduring, and after the intervention. Results suggest that passive GenAI agents\nimprove comprehension similarly to data storytelling both during and after\nintervention. Notably, proactive GenAI agents significantly enhance\ncomprehension after intervention compared to both passive GenAI agents and\nstandalone data storytelling, regardless of participants' visualisation\nliteracy, indicating sustained improvements and learning.","PeriodicalId":501541,"journal":{"name":"arXiv - CS - Human-Computer Interaction","volume":"43 1","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2024-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"arXiv - CS - Human-Computer Interaction","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/arxiv-2409.11645","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 0

Abstract

Generative AI (GenAI) agents offer a potentially scalable approach to support comprehending complex data visualisations, a skill many individuals struggle with. While data storytelling has proven effective, there is little evidence regarding the comparative effectiveness of GenAI agents. To address this gap, we conducted a randomised controlled study with 141 participants to compare the effectiveness and efficiency of data dialogues facilitated by both passive (which simply answer participants' questions about visualisations) and proactive (infused with scaffolding questions to guide participants through visualisations) GenAI agents against data storytelling in enhancing their comprehension of data visualisations. Comprehension was measured before, during, and after the intervention. Results suggest that passive GenAI agents improve comprehension similarly to data storytelling both during and after intervention. Notably, proactive GenAI agents significantly enhance comprehension after intervention compared to both passive GenAI agents and standalone data storytelling, regardless of participants' visualisation literacy, indicating sustained improvements and learning.

查看原文本刊更多论文

从数据故事到对话：生成式人工智能代理和数据故事在增强数据可视化理解方面的随机对照试验

生成式人工智能（GenAI）代理提供了一种潜在的可扩展方法来支持理解复杂的数据可视化，这是许多人都在努力学习的一项技能。虽然讲数据故事已被证明是有效的，但有关 GenAI 代理的比较效果的证据却很少。为了弥补这一不足，我们对141名参与者进行了随机对照研究，比较了被动式（只回答参与者关于可视化的问题）和主动式（通过脚手架问题引导参与者完成可视化）GenAI代理与数据讲故事在增强参与者对数据可视化的理解方面的效果和效率。在干预前、干预中和干预后都对参与者的理解能力进行了测量。结果表明，在干预期间和干预之后，被动 GenAI 代理对理解能力的提高与数据讲故事相似。值得注意的是，与被动型 GenAI 代理和单独的数据讲故事相比，主动型 GenAI 代理在干预后显著提高了理解能力，而与参与者的可视化读写能力无关，这表明了持续的改进和学习。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

arXiv - CS - Human-Computer Interaction

自引率

0.00%

发文量