利用大脑信号比较点相关性判断和成对相关性判断

IF 2.8 2区 管理学 Q2 COMPUTER SCIENCE, INFORMATION SYSTEMS
Shuqi Zhu, Xiaohui Xie, Ziyi Ye, Qingyao Ai, Yiqun Liu
{"title":"利用大脑信号比较点相关性判断和成对相关性判断","authors":"Shuqi Zhu,&nbsp;Xiaohui Xie,&nbsp;Ziyi Ye,&nbsp;Qingyao Ai,&nbsp;Yiqun Liu","doi":"10.1002/asi.24936","DOIUrl":null,"url":null,"abstract":"<p>How to collect relevance judgment has long been an important problem in Information Retrieval (IR). A popular method is to collect relevance judgment in a point-wise manner, in which assessors examine and give an absolute relevance score for each item independently of the others. As an alternative, pair-wise relevance judgment, also named preference judgment, allows an assessor to compare two items side-by-side and express their preference for one over the other. Previous work has explored the differences between these two paradigms of relevance judgments from many different aspects. Most of these works are conducted through explicit/implicit feedback. However, few works investigate the underlying neurological mechanisms of the two paradigms. In this paper, we conduct a lab study to investigate and compare point-wise and pair-wise relevance judgment in image search scenarios. We study the neurological mechanisms of the two paradigms through an event-related potential (ERP) analysis of the users' brain signals while viewing images during a search process. We have obtained several observations, such as search engine users tend to pay more attention to preferred items in the point-wise paradigm but unpreferred items in the pair-wise paradigm. Furthermore, we test the adoption of brain signals as implicit feedback for predicting pair-wise relevance judgment, highlighting the feasibility of leveraging brain signals to understand users' relevance judgments.</p>","PeriodicalId":48810,"journal":{"name":"Journal of the Association for Information Science and Technology","volume":"75 9","pages":"957-971"},"PeriodicalIF":2.8000,"publicationDate":"2024-06-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Comparing point-wise and pair-wise relevance judgment with brain signals\",\"authors\":\"Shuqi Zhu,&nbsp;Xiaohui Xie,&nbsp;Ziyi Ye,&nbsp;Qingyao Ai,&nbsp;Yiqun Liu\",\"doi\":\"10.1002/asi.24936\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p>How to collect relevance judgment has long been an important problem in Information Retrieval (IR). A popular method is to collect relevance judgment in a point-wise manner, in which assessors examine and give an absolute relevance score for each item independently of the others. As an alternative, pair-wise relevance judgment, also named preference judgment, allows an assessor to compare two items side-by-side and express their preference for one over the other. Previous work has explored the differences between these two paradigms of relevance judgments from many different aspects. Most of these works are conducted through explicit/implicit feedback. However, few works investigate the underlying neurological mechanisms of the two paradigms. In this paper, we conduct a lab study to investigate and compare point-wise and pair-wise relevance judgment in image search scenarios. We study the neurological mechanisms of the two paradigms through an event-related potential (ERP) analysis of the users' brain signals while viewing images during a search process. We have obtained several observations, such as search engine users tend to pay more attention to preferred items in the point-wise paradigm but unpreferred items in the pair-wise paradigm. Furthermore, we test the adoption of brain signals as implicit feedback for predicting pair-wise relevance judgment, highlighting the feasibility of leveraging brain signals to understand users' relevance judgments.</p>\",\"PeriodicalId\":48810,\"journal\":{\"name\":\"Journal of the Association for Information Science and Technology\",\"volume\":\"75 9\",\"pages\":\"957-971\"},\"PeriodicalIF\":2.8000,\"publicationDate\":\"2024-06-18\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Journal of the Association for Information Science and Technology\",\"FirstCategoryId\":\"91\",\"ListUrlMain\":\"https://onlinelibrary.wiley.com/doi/10.1002/asi.24936\",\"RegionNum\":2,\"RegionCategory\":\"管理学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"COMPUTER SCIENCE, INFORMATION SYSTEMS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of the Association for Information Science and Technology","FirstCategoryId":"91","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1002/asi.24936","RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}
引用次数: 0

摘要

长期以来,如何收集相关性判断一直是信息检索(IR)领域的一个重要问题。一种流行的方法是以点对点的方式收集相关性判断,即评估者对每个项目进行检查并给出一个独立于其他项目的绝对相关性分数。作为一种替代方法,成对相关性判断(也称为偏好判断)允许评估者并排比较两个项目,并表达他们对其中一个项目的偏好。以往的工作从许多不同方面探讨了这两种相关性判断范式之间的差异。这些研究大多是通过显性/隐性反馈进行的。然而,很少有人研究这两种范式的潜在神经机制。在本文中,我们进行了一项实验室研究,对图像搜索场景中的点式相关性判断和成对相关性判断进行了调查和比较。我们通过对用户在搜索过程中浏览图像时的大脑信号进行事件相关电位(ERP)分析,研究了两种范式的神经机制。我们获得了一些观察结果,如搜索引擎用户在点-线范式中倾向于更多地关注偏好项,而在对-线范式中则倾向于更多地关注非偏好项。此外,我们还测试了采用大脑信号作为隐式反馈来预测成对相关性判断的方法,从而凸显了利用大脑信号来理解用户相关性判断的可行性。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Comparing point-wise and pair-wise relevance judgment with brain signals

How to collect relevance judgment has long been an important problem in Information Retrieval (IR). A popular method is to collect relevance judgment in a point-wise manner, in which assessors examine and give an absolute relevance score for each item independently of the others. As an alternative, pair-wise relevance judgment, also named preference judgment, allows an assessor to compare two items side-by-side and express their preference for one over the other. Previous work has explored the differences between these two paradigms of relevance judgments from many different aspects. Most of these works are conducted through explicit/implicit feedback. However, few works investigate the underlying neurological mechanisms of the two paradigms. In this paper, we conduct a lab study to investigate and compare point-wise and pair-wise relevance judgment in image search scenarios. We study the neurological mechanisms of the two paradigms through an event-related potential (ERP) analysis of the users' brain signals while viewing images during a search process. We have obtained several observations, such as search engine users tend to pay more attention to preferred items in the point-wise paradigm but unpreferred items in the pair-wise paradigm. Furthermore, we test the adoption of brain signals as implicit feedback for predicting pair-wise relevance judgment, highlighting the feasibility of leveraging brain signals to understand users' relevance judgments.

求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
CiteScore
8.30
自引率
8.60%
发文量
115
期刊介绍: The Journal of the Association for Information Science and Technology (JASIST) is a leading international forum for peer-reviewed research in information science. For more than half a century, JASIST has provided intellectual leadership by publishing original research that focuses on the production, discovery, recording, storage, representation, retrieval, presentation, manipulation, dissemination, use, and evaluation of information and on the tools and techniques associated with these processes. The Journal welcomes rigorous work of an empirical, experimental, ethnographic, conceptual, historical, socio-technical, policy-analytic, or critical-theoretical nature. JASIST also commissions in-depth review articles (“Advances in Information Science”) and reviews of print and other media.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信