Grannies, tanning beds, tattoos and NASCAR: evaluation of search tasks with varying levels of cognitive complexity

Wan-Ching Wu, D. Kelly, A. Edwards, Jaime Arguello
{"title":"Grannies, tanning beds, tattoos and NASCAR: evaluation of search tasks with varying levels of cognitive complexity","authors":"Wan-Ching Wu, D. Kelly, A. Edwards, Jaime Arguello","doi":"10.1145/2362724.2362768","DOIUrl":null,"url":null,"abstract":"One of the most challenging aspects of designing an interactive information retrieval (IIR) study is the development of search tasks. In this paper, we present preliminary results of a study designed to evaluate a set of search tasks that were developed for use in IIR studies. We created 20 search tasks using five levels of cognitive complexity and four domains, and conducted a laboratory evaluation of these tasks with 48 undergraduate subjects. We describe preliminary results from an analysis of data from 24 subjects for 10 search tasks. Initial results show that, in general, as cognitive complexity increased, subjects issued more queries, clicked on more search results, viewed more URLs and took more time to complete the task. Subjects' expected and experienced difficulty ratings of tasks generally increased as cognitive complexity increased with some exceptions. When subjects were asked to rank tasks according to difficulty and engagement, tasks with higher cognitive complexity were rated as more difficult than tasks with lower cognitive complexity, but not necessarily as more engaging. These preliminary results suggest that behaviors and ratings are fairly consistent with the differences one might expect among the search tasks and provide initial evidence of the usefulness of these tasks in IIR studies.","PeriodicalId":413481,"journal":{"name":"International Conference on Information Interaction in Context","volume":"22 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2012-08-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"72","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"International Conference on Information Interaction in Context","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1145/2362724.2362768","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 72

Abstract

One of the most challenging aspects of designing an interactive information retrieval (IIR) study is the development of search tasks. In this paper, we present preliminary results of a study designed to evaluate a set of search tasks that were developed for use in IIR studies. We created 20 search tasks using five levels of cognitive complexity and four domains, and conducted a laboratory evaluation of these tasks with 48 undergraduate subjects. We describe preliminary results from an analysis of data from 24 subjects for 10 search tasks. Initial results show that, in general, as cognitive complexity increased, subjects issued more queries, clicked on more search results, viewed more URLs and took more time to complete the task. Subjects' expected and experienced difficulty ratings of tasks generally increased as cognitive complexity increased with some exceptions. When subjects were asked to rank tasks according to difficulty and engagement, tasks with higher cognitive complexity were rated as more difficult than tasks with lower cognitive complexity, but not necessarily as more engaging. These preliminary results suggest that behaviors and ratings are fairly consistent with the differences one might expect among the search tasks and provide initial evidence of the usefulness of these tasks in IIR studies.
奶奶、晒黑床、纹身和纳斯卡:对不同认知复杂程度的搜索任务的评估
设计交互式信息检索(IIR)研究最具挑战性的方面之一是搜索任务的开发。在本文中,我们提出了一项研究的初步结果,该研究旨在评估一组用于IIR研究的搜索任务。我们利用认知复杂性的五个层次和四个领域创建了20个搜索任务,并对48名本科生进行了实验室评估。我们描述了从10个搜索任务的24个受试者的数据分析的初步结果。初步结果显示,总体而言,随着认知复杂性的增加,受试者发出更多的查询,点击更多的搜索结果,查看更多的url,并花费更多的时间来完成任务。随着认知复杂性的增加,受试者对任务的预期和经历难度评分普遍增加,但也有例外。当受试者被要求根据难度和参与度对任务进行排序时,认知复杂性较高的任务被认为比认知复杂性较低的任务更难,但不一定更吸引人。这些初步结果表明,行为和评级与人们可能期望的搜索任务之间的差异相当一致,并为这些任务在IIR研究中的有用性提供了初步证据。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信