基于人工智能的高等教育在线团队合作自动检测

IF 2.9 3区 教育学 Q2 COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS
Alejandra J. Magana;Syed Tanzim Mubarrat;Dominic Kao;Bedrich Benes
{"title":"基于人工智能的高等教育在线团队合作自动检测","authors":"Alejandra J. Magana;Syed Tanzim Mubarrat;Dominic Kao;Bedrich Benes","doi":"10.1109/TLT.2024.3456447","DOIUrl":null,"url":null,"abstract":"Fostering productive engagement within teams has been found to improve student learning outcomes. Consequently, characterizing productive and unproductive time during teamwork sessions is a critical preliminary step to increase engagement in teamwork meetings. However, research from the cognitive sciences has mainly focused on characterizing levels of productive engagement. Thus, the theoretical contribution of this study focuses on characterizing active and passive forms of engagement, as well as negative and positive forms of engagement. In tandem, researchers have used computer-based methods to supplement quantitative and qualitative analyses to investigate teamwork engagement. Yet, these studies have been limited to information extracted primarily from one data stream. For instance, text data from discussion forums or video data from recordings. We developed an artificial intelligence (AI)-based automatic system that detects productive and unproductive engagement during live teamwork sessions. The technical contribution of this study focuses on the use of three data streams from an interactive session: audio, video, and text. We automatically analyze them and determine each team's level of engagement, such as productive engagement, unproductive engagement, disengagement, and idle. The AI-based system was validated based on hand-coded data. We used the system to characterize productive and unproductive engagement patterns in teams using deep learning methods. Results showed that there were \n<inline-formula><tex-math>$&gt;$</tex-math></inline-formula>\n91% prediction accuracy and \n<inline-formula><tex-math>$&lt; $</tex-math></inline-formula>\n7% mismatches between predictions for the three engagement detectors. Moreover, Pearson's \n<inline-formula><tex-math>$r$</tex-math></inline-formula>\n values between the predictions of the three detectors were \n<inline-formula><tex-math>$&gt;$</tex-math></inline-formula>\n0.844. On a scale of \n<inline-formula><tex-math>$-$</tex-math></inline-formula>\n1 (unproductive engagement) to 1 (productive engagement), the scores for all teams were 0.94 \n<inline-formula><tex-math>$\\pm$</tex-math></inline-formula>\n 0.04, suggesting high productive engagement. In addition, teams tended to mostly be in productive engagement before transitioning to disengagement (\n<inline-formula><tex-math>$&gt;$</tex-math></inline-formula>\n90.34% of the time) and to idle (\n<inline-formula><tex-math>$&gt;$</tex-math></inline-formula>\n93.69% of the time). Before transitioning to productive engagement, we noticed almost equal fractions of teams being in idle and disengagement modes. These results show that the system effectively detects engagement and can be a viable tool for characterizing productive and unproductive engagement patterns in teamwork sessions.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"2091-2106"},"PeriodicalIF":2.9000,"publicationDate":"2024-09-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10669806","citationCount":"0","resultStr":"{\"title\":\"AI-Based Automatic Detection of Online Teamwork Engagement in Higher Education\",\"authors\":\"Alejandra J. Magana;Syed Tanzim Mubarrat;Dominic Kao;Bedrich Benes\",\"doi\":\"10.1109/TLT.2024.3456447\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Fostering productive engagement within teams has been found to improve student learning outcomes. Consequently, characterizing productive and unproductive time during teamwork sessions is a critical preliminary step to increase engagement in teamwork meetings. However, research from the cognitive sciences has mainly focused on characterizing levels of productive engagement. Thus, the theoretical contribution of this study focuses on characterizing active and passive forms of engagement, as well as negative and positive forms of engagement. In tandem, researchers have used computer-based methods to supplement quantitative and qualitative analyses to investigate teamwork engagement. Yet, these studies have been limited to information extracted primarily from one data stream. For instance, text data from discussion forums or video data from recordings. We developed an artificial intelligence (AI)-based automatic system that detects productive and unproductive engagement during live teamwork sessions. The technical contribution of this study focuses on the use of three data streams from an interactive session: audio, video, and text. We automatically analyze them and determine each team's level of engagement, such as productive engagement, unproductive engagement, disengagement, and idle. The AI-based system was validated based on hand-coded data. We used the system to characterize productive and unproductive engagement patterns in teams using deep learning methods. Results showed that there were \\n<inline-formula><tex-math>$&gt;$</tex-math></inline-formula>\\n91% prediction accuracy and \\n<inline-formula><tex-math>$&lt; $</tex-math></inline-formula>\\n7% mismatches between predictions for the three engagement detectors. Moreover, Pearson's \\n<inline-formula><tex-math>$r$</tex-math></inline-formula>\\n values between the predictions of the three detectors were \\n<inline-formula><tex-math>$&gt;$</tex-math></inline-formula>\\n0.844. On a scale of \\n<inline-formula><tex-math>$-$</tex-math></inline-formula>\\n1 (unproductive engagement) to 1 (productive engagement), the scores for all teams were 0.94 \\n<inline-formula><tex-math>$\\\\pm$</tex-math></inline-formula>\\n 0.04, suggesting high productive engagement. In addition, teams tended to mostly be in productive engagement before transitioning to disengagement (\\n<inline-formula><tex-math>$&gt;$</tex-math></inline-formula>\\n90.34% of the time) and to idle (\\n<inline-formula><tex-math>$&gt;$</tex-math></inline-formula>\\n93.69% of the time). Before transitioning to productive engagement, we noticed almost equal fractions of teams being in idle and disengagement modes. These results show that the system effectively detects engagement and can be a viable tool for characterizing productive and unproductive engagement patterns in teamwork sessions.\",\"PeriodicalId\":49191,\"journal\":{\"name\":\"IEEE Transactions on Learning Technologies\",\"volume\":\"17 \",\"pages\":\"2091-2106\"},\"PeriodicalIF\":2.9000,\"publicationDate\":\"2024-09-09\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10669806\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"IEEE Transactions on Learning Technologies\",\"FirstCategoryId\":\"95\",\"ListUrlMain\":\"https://ieeexplore.ieee.org/document/10669806/\",\"RegionNum\":3,\"RegionCategory\":\"教育学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Learning Technologies","FirstCategoryId":"95","ListUrlMain":"https://ieeexplore.ieee.org/document/10669806/","RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS","Score":null,"Total":0}
引用次数: 0

摘要

研究发现,在团队中培养富有成效的参与能提高学生的学习成绩。因此,确定团队合作会议期间有成效和无成效时间的特征,是提高团队合作会议参与度的关键第一步。然而,认知科学的研究主要集中在描述生产性参与的水平。因此,本研究的理论贡献主要集中在描述主动和被动的参与形式,以及消极和积极的参与形式。与此同时,研究人员还使用基于计算机的方法来补充定量和定性分析,以调查团队合作参与度。然而,这些研究主要局限于从一种数据流中提取信息。例如,来自论坛的文本数据或来自录音的视频数据。我们开发了一种基于人工智能(AI)的自动系统,可以检测现场团队合作会议中的生产性参与和非生产性参与。本研究的技术贡献集中在使用互动会议的三个数据流:音频、视频和文本。我们对它们进行自动分析,并确定每个团队的参与程度,如生产性参与、非生产性参与、脱离参与和闲置参与。基于人工智能的系统根据手工编码的数据进行了验证。我们利用该系统,采用深度学习方法来描述团队中的生产性参与和非生产性参与模式。结果表明,三种参与度检测器的预测准确率为91%,预测不匹配率为7%。此外,三种检测器预测值之间的皮尔逊r值为$>0.844。在$-$1(非生产性参与)到$1(生产性参与)的范围内,所有团队的得分均为 0.94 $\pm$ 0.04,表明生产性参与程度较高。此外,在过渡到脱离($>90.34% 的时间)和闲置($>93.69% 的时间)之前,团队往往大多处于生产性参与状态。在过渡到生产性参与之前,我们注意到处于闲置和脱离模式的团队比例几乎相等。这些结果表明,该系统能有效检测参与情况,并可作为一种可行的工具,用于描述团队工作会议中的生产性参与和非生产性参与模式。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
AI-Based Automatic Detection of Online Teamwork Engagement in Higher Education
Fostering productive engagement within teams has been found to improve student learning outcomes. Consequently, characterizing productive and unproductive time during teamwork sessions is a critical preliminary step to increase engagement in teamwork meetings. However, research from the cognitive sciences has mainly focused on characterizing levels of productive engagement. Thus, the theoretical contribution of this study focuses on characterizing active and passive forms of engagement, as well as negative and positive forms of engagement. In tandem, researchers have used computer-based methods to supplement quantitative and qualitative analyses to investigate teamwork engagement. Yet, these studies have been limited to information extracted primarily from one data stream. For instance, text data from discussion forums or video data from recordings. We developed an artificial intelligence (AI)-based automatic system that detects productive and unproductive engagement during live teamwork sessions. The technical contribution of this study focuses on the use of three data streams from an interactive session: audio, video, and text. We automatically analyze them and determine each team's level of engagement, such as productive engagement, unproductive engagement, disengagement, and idle. The AI-based system was validated based on hand-coded data. We used the system to characterize productive and unproductive engagement patterns in teams using deep learning methods. Results showed that there were $>$ 91% prediction accuracy and $< $ 7% mismatches between predictions for the three engagement detectors. Moreover, Pearson's $r$ values between the predictions of the three detectors were $>$ 0.844. On a scale of $-$ 1 (unproductive engagement) to 1 (productive engagement), the scores for all teams were 0.94 $\pm$ 0.04, suggesting high productive engagement. In addition, teams tended to mostly be in productive engagement before transitioning to disengagement ( $>$ 90.34% of the time) and to idle ( $>$ 93.69% of the time). Before transitioning to productive engagement, we noticed almost equal fractions of teams being in idle and disengagement modes. These results show that the system effectively detects engagement and can be a viable tool for characterizing productive and unproductive engagement patterns in teamwork sessions.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
IEEE Transactions on Learning Technologies
IEEE Transactions on Learning Technologies COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS-
CiteScore
7.50
自引率
5.40%
发文量
82
审稿时长
>12 weeks
期刊介绍: The IEEE Transactions on Learning Technologies covers all advances in learning technologies and their applications, including but not limited to the following topics: innovative online learning systems; intelligent tutors; educational games; simulation systems for education and training; collaborative learning tools; learning with mobile devices; wearable devices and interfaces for learning; personalized and adaptive learning systems; tools for formative and summative assessment; tools for learning analytics and educational data mining; ontologies for learning systems; standards and web services that support learning; authoring tools for learning materials; computer support for peer tutoring; learning via computer-mediated inquiry, field, and lab work; social learning techniques; social networks and infrastructures for learning and knowledge sharing; and creation and management of learning objects.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信