发布求助
文献互助 智能选刊 最新文献

Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization最新文献

筛选
英文 中文
More bang for your research buck: toward recommender systems for visual analytics
Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization Pub Date : 2014-11-10 DOI: 10.1145/2669557.2669566
L. Blaha, Dustin L. Arendt, Fairul Mohd-Zaid
{"title":"More bang for your research buck: toward recommender systems for visual analytics","authors":"L. Blaha, Dustin L. Arendt, Fairul Mohd-Zaid","doi":"10.1145/2669557.2669566","DOIUrl":"https://doi.org/10.1145/2669557.2669566","url":null,"abstract":"We propose a set of common sense steps required to develop a recommender system for visual analytics. Such a system is an essential way to get additional mileage out of costly user studies, which are typically archived post publication. Crucially, we propose conducting user studies in a manner that allows machine learning techniques to elucidate relationships between experimental data (i.e., user performance) and metrics about the data being visualized and candidate visual representations. We execute a case study within our framework to extract simple rules of thumb that relate different data metrics and visualization characteristics to patterns of user errors on several network analysis tasks. Our case study suggests a research agenda supporting the development of general, robust visualization recommender systems.","PeriodicalId":179584,"journal":{"name":"Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2014-11-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131552571","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 5
Just the other side of the coin?: from error- to insight-analysis 只是硬币的另一面吗?从错误分析到洞察分析
Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization Pub Date : 2014-11-10 DOI: 10.1145/2669557.2669570
M. Smuc
{"title":"Just the other side of the coin?: from error- to insight-analysis","authors":"M. Smuc","doi":"10.1145/2669557.2669570","DOIUrl":"https://doi.org/10.1145/2669557.2669570","url":null,"abstract":"To shed more light on data explorers dealing with complex information visualizations in real world scenarios, new methodologies and models are needed which overcome existing explanatory gaps. Therefore, a novel model to analyze users' errors and insights is outlined that is derived from Rasmussen's model on different levels of cognitive processing, and integrates explorers' skills, schemes, and knowledge. After locating this model in the landscape of theories for visual analytics, the main building blocks of the model, where three cognitive processing levels are interlinked, are described in detail. Finally, its applicability, challenges in measurement and future research options are discussed.","PeriodicalId":179584,"journal":{"name":"Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2014-11-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132172862","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
Sanity check for class-coloring-based evaluation of dimension reduction techniques 基于类颜色的降维技术评估的完整性检查
Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization Pub Date : 2014-11-10 DOI: 10.1145/2669557.2669578
Michaël Aupetit
{"title":"Sanity check for class-coloring-based evaluation of dimension reduction techniques","authors":"Michaël Aupetit","doi":"10.1145/2669557.2669578","DOIUrl":"https://doi.org/10.1145/2669557.2669578","url":null,"abstract":"Dimension Reduction techniques used to visualize multidimensional data provide a scatterplot spatialization of data similarities. A widespread way to evaluate the quality of such DR techniques is to use labeled data as a ground truth and to call the reader as a witness to qualify the visualization by looking at class-cluster correlations within the scatterplot. We expose the pitfalls of this evaluation process and we propose a principled solution to guide researchers to improve the way they use this visual evaluation of DR techniques.","PeriodicalId":179584,"journal":{"name":"Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2014-11-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116894669","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 20
Towards analyzing eye tracking data for evaluating interactive visualization systems 分析眼动追踪数据以评估交互式可视化系统
Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization Pub Date : 2014-11-10 DOI: 10.1145/2669557.2669569
Tanja Blascheck, T. Ertl
{"title":"Towards analyzing eye tracking data for evaluating interactive visualization systems","authors":"Tanja Blascheck, T. Ertl","doi":"10.1145/2669557.2669569","DOIUrl":"https://doi.org/10.1145/2669557.2669569","url":null,"abstract":"Eye tracking can be a suitable evaluation method for determining which regions and objects of a stimulus a human viewer perceived. Analysts can use eye tracking as a complement to other evaluation methods for a more holistic assessment of novel visualization techniques beyond time and error measures. Up to now, most stimuli in eye tracking are either static stimuli or videos. Since interaction is an integral part of visualization, an evaluation should include interaction. In this paper, we present an extensive literature review on evaluation methods for interactive visualizations. Based on the literature review we propose ideas for analyzing eye movement data from interactive stimuli. This requires looking critically at challenges induced by interactive stimuli. The first step is to collect data using different study methods. In our case, we look at using eye tracking, interaction logs, and thinking-aloud protocols. In addition, this requires a thorough synchronization of the mentioned study methods. To analyze the collected data new analysis techniques have to be developed. We investigate existing approaches and how we can adapt them to new data types as well as sketch ideas how new approaches can look like.","PeriodicalId":179584,"journal":{"name":"Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2014-11-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"120940303","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 13
Repeated measures design in crowdsourcing-based experiments for visualization 基于众包的可视化实验中的重复测量设计
Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization Pub Date : 2014-11-10 DOI: 10.1145/2669557.2669561
A. Abdul-Rahman, Karl J. Proctor, Brian Duffy, Min Chen
{"title":"Repeated measures design in crowdsourcing-based experiments for visualization","authors":"A. Abdul-Rahman, Karl J. Proctor, Brian Duffy, Min Chen","doi":"10.1145/2669557.2669561","DOIUrl":"https://doi.org/10.1145/2669557.2669561","url":null,"abstract":"Crowdsourcing platforms, such as Amazon's Mechanical Turk (MTurk), are providing visualization researchers with a new avenue for conducting empirical studies. While such platforms offer several advantages over lab-based studies, they also feature some \"unknown\" or \"uncontrolled\" variables, which could potentially introduce serious confounding effects in the resultant measurement data. In this paper, we present our experience of using repeated measures in three empirical studies using MTurk. Each study presented participants with a set of stimuli, each featuring a condition of an independent variable. Participants were exposed to stimuli repeatedly in a pseudo-random order through four trials and their responses were measured digitally. Only a small portion of the participants were able to perform with absolute consistency for all stimuli throughout each experiment. This suggests that a repeated measures design is highly desirable (if not essential) when designing empirical studies for crowdsourcing platforms. Additionally, the majority of participants performed their tasks with reasonable consistency when all stimuli in an experiment are considered collectively. In other words, to most participants, inconsistency occurred occasionally. This suggests that crowdsourcing remains a valid experimental environment, provided that one can integrate the means to observe and alleviate the potential confounding effects of \"unknown\" or \"uncontrolled\" variables in the design of the experiment.","PeriodicalId":179584,"journal":{"name":"Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2014-11-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127934238","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 7
Toward visualization-specific heuristic evaluation 面向可视化的启发式评估
Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization Pub Date : 2014-11-10 DOI: 10.1145/2669557.2669580
Alvin E Tarrell, Ann L. Fruhling, R. Borgo, C. Forsell, G. Grinstein, J. Scholtz
{"title":"Toward visualization-specific heuristic evaluation","authors":"Alvin E Tarrell, Ann L. Fruhling, R. Borgo, C. Forsell, G. Grinstein, J. Scholtz","doi":"10.1145/2669557.2669580","DOIUrl":"https://doi.org/10.1145/2669557.2669580","url":null,"abstract":"This position paper describes heuristic evaluation as it relates to visualization and visual analytics. We review heuristic evaluation in general, then comment on previous process-based, performance-based, and framework-based efforts to adapt the method to visualization-specific needs. We postulate that the framework-based approach holds the most promise for future progress in development of visualization-specific heuristics, and propose a specific framework as a starting point. We then recommend a method for community involvement and input into the further development of the heuristic framework and more detailed design and evaluation guidelines.","PeriodicalId":179584,"journal":{"name":"Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2014-11-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114277073","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 18
Utility evaluation of models 模型效用评价
Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization Pub Date : 2014-11-10 DOI: 10.1145/2669557.2669562
J. Scholtz, Oriana Love, M. Whiting, Duncan Hodges, Lia Emanuel, D. Fraser
{"title":"Utility evaluation of models","authors":"J. Scholtz, Oriana Love, M. Whiting, Duncan Hodges, Lia Emanuel, D. Fraser","doi":"10.1145/2669557.2669562","DOIUrl":"https://doi.org/10.1145/2669557.2669562","url":null,"abstract":"In this paper, we present three case studies of utility evaluations of underlying models in software systems: a user-model, technical and social models both singly and in combination, and a research-based model for user identification. Each of the three cases used a different approach to evaluating the model and each had challenges to overcome in designing and implementing the evaluation. We describe the methods we used and challenges faced in designing the evaluation procedures, summarize the lessons learned, enumerate considerations for those undertaking such evaluations, and present directions for future work.","PeriodicalId":179584,"journal":{"name":"Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2014-11-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129461773","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
User tasks for evaluation: untangling the terminology throughout visualization design and development 评估的用户任务:理清可视化设计和开发中的术语
Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization Pub Date : 2014-11-10 DOI: 10.1145/2669557.2669568
A. Rind, W. Aigner, Markus Wagner, S. Miksch, T. Lammarsch
{"title":"User tasks for evaluation: untangling the terminology throughout visualization design and development","authors":"A. Rind, W. Aigner, Markus Wagner, S. Miksch, T. Lammarsch","doi":"10.1145/2669557.2669568","DOIUrl":"https://doi.org/10.1145/2669557.2669568","url":null,"abstract":"User tasks play a pivotal role in evaluation throughout visualization design and development. However, the term 'task' is used ambiguously within the visualization community. In this position paper, we critically analyze the relevant literature and systematically compare definitions for 'task' and the usage of related terminology. In doing so, we identify a three-dimensional conceptual space of user tasks in visualization. Using these dimensions, visualization researchers can better formulate their contributions which helps advance visualization as a whole.","PeriodicalId":179584,"journal":{"name":"Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2014-11-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115894333","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 7
Evaluation of information visualization techniques: analysing user experience with reaction cards 评价信息可视化技术:用反应卡分析用户体验
Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization Pub Date : 2014-11-10 DOI: 10.1145/2669557.2669565
Tanja Mercun
{"title":"Evaluation of information visualization techniques: analysing user experience with reaction cards","authors":"Tanja Mercun","doi":"10.1145/2669557.2669565","DOIUrl":"https://doi.org/10.1145/2669557.2669565","url":null,"abstract":"The paper originates from the idea that in the field of information visualization, positive user experience is extremely important if we wish to see users adopt and engage with the novel information visualization tools. Suggesting the use of product reaction card method to evaluate user experience, the paper gives an example of FrbrVis prototype to demonstrate how the results of this method could be analysed and used for comparing different designs. The authors also propose five dimensions of user experience (UX) that could be gathered from reaction cards and conclude that the results from reaction cards mirror and add to other performance and preference indicators.","PeriodicalId":179584,"journal":{"name":"Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2014-11-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122798829","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 14
Value-driven evaluation of visualizations 价值驱动的可视化评估
Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization Pub Date : 2014-11-10 DOI: 10.1145/2669557.2669579
J. Stasko
{"title":"Value-driven evaluation of visualizations","authors":"J. Stasko","doi":"10.1145/2669557.2669579","DOIUrl":"https://doi.org/10.1145/2669557.2669579","url":null,"abstract":"Existing evaluations of data visualizations often employ a series of low-level, detailed questions to be answered or benchmark tasks to be performed. While that methodology can be helpful to determine a visualization's usability, such evaluations overlook the key benefits that visualization uniquely provides over other data analysis methods. I propose a value-driven evaluation of visualizations in which a person illustrates a system's value through four important capabilities: minimizing the time to answer diverse questions, spurring the generation of insights and insightful questions, conveying the essence of the data, and generating confidence and knowledge about the data's domain and context. Additionally, I explain how interaction is instrumental in creating much of the value that can be found in visualizations.","PeriodicalId":179584,"journal":{"name":"Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2014-11-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115080941","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 50
下一页 尾页
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整，为了账户安全，请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
请完成安全验证×
相关产品
×
本文献相关产品
联系我们：info@booksci.cn Book学术提供免费学术资源搜索服务，方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号：481959085
Book学术官方微信