Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization最新文献_第2页

Field experiment methodology for pair analytics 配对分析的现场实验方法

Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization Pub Date : 2014-11-10 DOI: 10.1145/2669557.2669572

Linda T. Kaastra, Brian D. Fisher

引用次数: 24

Evaluation methodology for comparing memory and communication of analytic processes in visual analytics 在视觉分析中比较分析过程的记忆和交流的评价方法

Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization Pub Date : 2014-11-10 DOI: 10.1145/2669557.2669563

E. Ragan, J. Goodall

{"title":"Evaluation methodology for comparing memory and communication of analytic processes in visual analytics","authors":"E. Ragan, J. Goodall","doi":"10.1145/2669557.2669563","DOIUrl":"https://doi.org/10.1145/2669557.2669563","url":null,"abstract":"Provenance tools can help capture and represent the history of analytic processes. In addition to supporting analytic performance, provenance tools can be used to support memory of the process and communication of the steps to others. Objective evaluation methods are needed to evaluate how well provenance tools support analysts' memory and communication of analytic processes. In this paper, we present several methods for the evaluation of process memory, and we discuss the advantages and limitations of each. We discuss methods for determining a baseline process for comparison, and we describe various methods that can be used to elicit memory of an analysis for evaluation. Additionally, we discuss methods for conducting quantitative and qualitative analyses of process memory. We discuss the methodology in the context of a case study in using the evaluation methods for a user study. By organizing possible memory evaluation methods and providing a meta-analysis of the potential benefits and drawbacks of different approaches, this paper can inform study design and encourage objective evaluation of process memory and communication.","PeriodicalId":179584,"journal":{"name":"Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2014-11-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128068611","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 10

Oopsy-daisy: failure stories in quantitative evaluation studies for visualizations 可怜的雏菊:用于可视化的定量评估研究中的失败故事

Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization Pub Date : 2014-11-10 DOI: 10.1145/2669557.2669576

Sung-Hee Kim, Ji Soo Yi, N. Elmqvist

{"title":"Oopsy-daisy: failure stories in quantitative evaluation studies for visualizations","authors":"Sung-Hee Kim, Ji Soo Yi, N. Elmqvist","doi":"10.1145/2669557.2669576","DOIUrl":"https://doi.org/10.1145/2669557.2669576","url":null,"abstract":"Designing, conducting, and interpreting evaluation studies with human participants is challenging. While researchers in cognitive psychology, social science, and human-computer interaction view competence in evaluation study methodology a key job skill, it is only recently that visualization researchers have begun to feel the need to learn this skill as well. Acquiring such competence is a lengthy and difficult process fraught with much trial and error. Recent work on patterns for visualization evaluation is now providing much-needed best practices for how to evaluate a visualization technique with human participants. However, negative examples of evaluation methods that fail, yield no usable results, or simply do not work are still missing, mainly because of the difficulty and lack of incentive for publishing negative results or failed research. In this paper, we take the position that there are many good ideas with the best intentions for how to evaluate a visualization tool that simply do not work. We call upon the community to help collect these negative examples in order to show the other side of the coin: what not to do when trying to evaluate visualization.","PeriodicalId":179584,"journal":{"name":"Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2014-11-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123727678","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 5

Navigating reductionism and holism in evaluation 评价中的还原论与整体论

Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization Pub Date : 2014-11-10 DOI: 10.1145/2669557.2669577

M. Correll, E. Alexander, Danielle Albers, Alper Sarikaya, Michael Gleicher

{"title":"Navigating reductionism and holism in evaluation","authors":"M. Correll, E. Alexander, Danielle Albers, Alper Sarikaya, Michael Gleicher","doi":"10.1145/2669557.2669577","DOIUrl":"https://doi.org/10.1145/2669557.2669577","url":null,"abstract":"In this position paper, we enumerate two approaches to the evaluation of visualizations which are associated with two approaches to knowledge formation in science: reductionism, which holds that the understanding of complex phenomena is based on the understanding of simpler components; and holism, which states that complex phenomena have characteristics more than the sum of their parts and must be understood as complete, irreducible units. While we believe that each approach has benefits for evaluating visualizations, we claim that strict adherence to one perspective or the other can make it difficult to generate a full evaluative picture of visualization tools and techniques. We argue for movement between and among these perspectives in order to generate knowledge that is both grounded (i.e. its constituent parts work) and validated (i.e. the whole operates correctly). We conclude with examples of techniques which we believe represent movements of this sort from our own work, highlighting areas where we have both \"built up\" reductionist techniques into larger contexts, and \"broken down\" holistic techniques to create generalizable knowledge.","PeriodicalId":179584,"journal":{"name":"Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2014-11-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128163835","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 8

Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization 第五届超越时间与错误:视觉化的新评估方法研讨会论文集

Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization Pub Date : 2014-11-10 DOI: 10.1145/2669557

Heidi Lam, Petra Isenberg, Tobias Isenberg, M. Sedlmair

{"title":"Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization","authors":"Heidi Lam, Petra Isenberg, Tobias Isenberg, M. Sedlmair","doi":"10.1145/2669557","DOIUrl":"https://doi.org/10.1145/2669557","url":null,"abstract":"Visualization has shown its ability to produce powerful tools for analyzing, understanding, and communicating data and making it accessible for several different tasks and purposes. Impact of visualization to everyday work and personal lives is demonstrated by many successes stories---such as the increasing prevalence of Tableau, the interactive visualizations produced by the New York Times, or toolkits like VTK/Paraview to name just a few. A large community of casual and professional users are increasingly consuming and producing both interactive and static visualizations.While interactive visualizations move from research into practice at an increasing rate, it still remains an important challenge to find appropriate methods to evaluate their utility and usability. There is a growing need in the community to develop special approaches and metrics for evaluation at all stages of the development life cycle that address specific needs in visualization. This need is reflected, for example, in the increasing number of papers on visualization evaluation---not just at BELIV but also in other venues such as the IEEE VIS conferences and EuroVis. The goal of the BELIV workshop is to continue to provide a dedicated event for discussing visualization evaluation and to spread the word on alternative and novel evaluation methods and methodologies in our community.","PeriodicalId":179584,"journal":{"name":"Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2014-11-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130862559","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 15

Visualizing dimensionally-reduced data: interviews with analysts and a characterization of task sequences 可视化降维数据:与分析师的访谈和任务序列的表征

Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization Pub Date : 2014-11-10 DOI: 10.1145/2669557.2669559

M. Brehmer, M. Sedlmair, S. Ingram, T. Munzner

引用次数: 85

Evaluating visual analytics with eye tracking 用眼动追踪评估视觉分析

Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization Pub Date : 2014-11-10 DOI: 10.1145/2669557.2669560

K. Kurzhals, Brian D. Fisher, Michael Burch, D. Weiskopf

{"title":"Evaluating visual analytics with eye tracking","authors":"K. Kurzhals, Brian D. Fisher, Michael Burch, D. Weiskopf","doi":"10.1145/2669557.2669560","DOIUrl":"https://doi.org/10.1145/2669557.2669560","url":null,"abstract":"The application of eye tracking for the evaluation of humans' viewing behavior is a common approach in psychological research. So far, the use of this technique for the evaluation of visual analytics and visualization is less prominent. We investigate recent scientific publications from the main visualization and visual analytics conferences and journals that include an evaluation by eye tracking. Furthermore, we provide an overview of evaluation goals that can be achieved by eye tracking and state-of-the-art analysis techniques for eye tracking data. Ideally, visual analytics leads to a mixed-initiative cognitive system where the mechanism of distribution is the interaction of the user with visualization environments. Therefore, we also include a discussion of cognitive approaches and models to include the user in the evaluation process. Based on our review of the current use of eye tracking evaluation in our field and the cognitive theory, we propose directions of future research on evaluation methodology, leading to the grand challenge of developing an evaluation approach to the mixed-initiative cognitive system of visual analytics.","PeriodicalId":179584,"journal":{"name":"Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2014-11-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132221769","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 67

Considerations for characterizing domain problems 描述领域问题的注意事项

Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization Pub Date : 2014-11-10 DOI: 10.1145/2669557.2669573

Kirsten M. Winters, D. Lach, J. Cushing

引用次数: 7

Experiences and challenges with evaluation methods in practice: a case study 评估方法在实践中的经验与挑战:个案研究

Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization Pub Date : 2014-11-10 DOI: 10.1145/2669557.2669571

Simone Kriglstein, M. Pohl, Nikolaus Suchy, J. Gärtner, T. Gschwandtner, S. Miksch

引用次数: 2

Evaluating user behavior and strategy during visual exploration 在视觉探索过程中评估用户行为和策略

Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization Pub Date : 2014-11-10 DOI: 10.1145/2669557.2669575

K. Reda, Andrew E. Johnson, J. Leigh, M. Papka

引用次数: 23