Dimitar Garkov, C. Müller, Matthias Braun, D. Weiskopf, Falk Schreiber
{"title":"Research Data Curation in Visualization : Position Paper","authors":"Dimitar Garkov, C. Müller, Matthias Braun, D. Weiskopf, Falk Schreiber","doi":"10.1109/BELIV57783.2022.00011","DOIUrl":"https://doi.org/10.1109/BELIV57783.2022.00011","url":null,"abstract":"Research data curation is the act of carefully preparing research data and artifacts for sharing and long-term preservation. Research data management is centrally implemented and formally defined in a data management plan to enable data curation. In tandem, data curation and management facilitate research repeatability. In contrast to other research fields, data curation and management in visualization are not yet part of the researcher’s compendium. In this position paper, we discuss the unique challenges visualization faces and propose how data curation can be practically realized. We share eight lessons learned in managing data in two large research consortia, outline the larger curation workflow, and define the typical roles. We complement our lessons with minimum criteria for selecting a suitable data repository and five challenging scenarios that occur in practice. We conclude with a vision of how the visualization research community can pave the way for new curation standards.","PeriodicalId":299298,"journal":{"name":"2022 IEEE Evaluation and Beyond - Methodological Approaches for Visualization (BELIV)","volume":"267 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114175614","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Tomás Alves, Carlota Dias, D. Gonçalves, S. Gama, J. Henriques-Calado
{"title":"How Personality and Visual Channels Affect Insight Generation","authors":"Tomás Alves, Carlota Dias, D. Gonçalves, S. Gama, J. Henriques-Calado","doi":"10.1109/BELIV57783.2022.00010","DOIUrl":"https://doi.org/10.1109/BELIV57783.2022.00010","url":null,"abstract":"Gaining insight is considered one of the relevant purposes of visual data exploration, yet studies that categorize insights are rare. This paper reports on a study to understand if the categorization model used to describe insights and personality factors affect insight-based evaluations’ findings. Participants completed a set of tasks with three hierarchical visualizations and then reported what insights they could gather from them. Results show that the insight categorization taxonomies produce different descriptions of insights based on the same corpus of responses. In addition, our findings suggest that the openness to experience trait positively influences the number of reported insights. Both these factors may create obstacles to the design of insight-based evaluations and, consequently, should be controlled in the experimental design. We discuss the study implications, lessons learned, and future work opportunities.","PeriodicalId":299298,"journal":{"name":"2022 IEEE Evaluation and Beyond - Methodological Approaches for Visualization (BELIV)","volume":"123 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129660023","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Power Overwhelming: Quantifying the Energy Cost of Visualisation","authors":"C. Müller, Moritz Heinemann, D. Weiskopf, T. Ertl","doi":"10.1109/BELIV57783.2022.00009","DOIUrl":"https://doi.org/10.1109/BELIV57783.2022.00009","url":null,"abstract":"GPUs are the power-hungry tool of many visualisation researchers. However, their energy consumption has mostly been investigated outside the visualisation community, albeit our algorithms can generate more complex workloads than compute kernels. Additionally, a raising number of web-based visualisations potentially makes consumers other than the GPU more relevant. We present measurement setups for quantifying the energy cost of visualisation, ranging from software sensors over external power meters and micro controller-based setups to using oscilloscopes. These setups cover energy consumption of GPUs, CPUs and other components of a computing system. Using raycasting of spherical glyphs, volume rendering and D3 visualisations as examples, we show that there are viable options for evaluating most kinds of visualisations. We conclude by stating the challenges to a broader application of these techniques and by making recommendations on how to overcome these.","PeriodicalId":299298,"journal":{"name":"2022 IEEE Evaluation and Beyond - Methodological Approaches for Visualization (BELIV)","volume":"37 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126030443","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Mai Elshehaly, Kuldeep Sohal, T. Lawton, M. Bryant, M. Mon-Williams
{"title":"Creative Visualisation Opportunities Workshops: A Case Study in Population Health","authors":"Mai Elshehaly, Kuldeep Sohal, T. Lawton, M. Bryant, M. Mon-Williams","doi":"10.1109/BELIV57783.2022.00006","DOIUrl":"https://doi.org/10.1109/BELIV57783.2022.00006","url":null,"abstract":"Population Health Management (PHM) relies on the analysis of data from several sources to account for the complex interaction of factors that contribute to the health and well-being of a population, while considering biases and inequalities across sub-populations. Visualisation is emerging as an essential tool for insight generation from data shared and linked across services including healthcare, education, housing, policing, etc. However, visualisation design is challenged by poor data connectivity and quality, high dimensionality and complexity of real-world routinely collected data, in addition to the heterogeneity of users’ backgrounds and tasks. The Creative Visualisation Opportunities (CVO) framework provides a structured approach for working with diverse communities of visualisation stakeholders and defines a set of participatory activities for the effective elicitation of requirements and visualisation design alternatives. We conducted three workshops, applying variations of the CVO framework, with over one hundred participants from the PHM domain, including clinicians, researchers, government and private sector representatives, and local communities. In this paper, we present the results of preliminary analysis of these activities and report on the perceived impact of visualisation in this domain from a stakeholders’ perspective. We report real-world successes and limitations of applying the framework in different formats (through online and in-person workshops), and reflect on lessons learned for task analysis and visualisation design in the PHM domain.","PeriodicalId":299298,"journal":{"name":"2022 IEEE Evaluation and Beyond - Methodological Approaches for Visualization (BELIV)","volume":"47 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128356109","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Hamza Elhamdadi, Aimen Gaba, Yea-Seul Kim, Cindy Xiong
{"title":"How Do We Measure Trust in Visual Data Communication?","authors":"Hamza Elhamdadi, Aimen Gaba, Yea-Seul Kim, Cindy Xiong","doi":"10.1109/BELIV57783.2022.00014","DOIUrl":"https://doi.org/10.1109/BELIV57783.2022.00014","url":null,"abstract":"Trust is fundamental to effective visual data communication between the visualization designer and the reader. Although personal experience and preference influence readers’ trust in visualizations, visualization designers can leverage design techniques to create visualizations that evoke a \"calibrated trust,\" at which readers arrive after critically evaluating the information presented. To systematically understand what drives readers to engage in \"calibrated trust,\" we must first equip ourselves with reliable and valid methods for measuring trust. Computer science and data visualization researchers have not yet reached a consensus on a trust definition or metric, which are essential to building a comprehensive trust model in human-data interaction. On the other hand, social scientists and behavioral economists have developed and perfected metrics that can measure generalized and interpersonal trust, which the visualization community can reference, modify, and adapt for our needs. In this paper, we gather existing methods for evaluating trust from other disciplines and discuss how we might use them to measure, define, and model trust in data visualization research. Specifically, we discuss quantitative surveys from social sciences, trust games from behavioral economics, measuring trust through measuring belief updating, and measuring trust through perceptual methods. We assess the potential issues with these methods and consider how we can systematically apply them to visualization research.","PeriodicalId":299298,"journal":{"name":"2022 IEEE Evaluation and Beyond - Methodological Approaches for Visualization (BELIV)","volume":"43 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-09-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121634661","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
K. Kucher, N. Sultanum, Angel Daza, Vasiliki Simaki, Maria Skeppstedt, Barbara Plank, Jean-Daniel Fekete, Narges Mahyar
{"title":"An Interdisciplinary Perspective on Evaluation and Experimental Design for Visual Text Analytics: Position Paper","authors":"K. Kucher, N. Sultanum, Angel Daza, Vasiliki Simaki, Maria Skeppstedt, Barbara Plank, Jean-Daniel Fekete, Narges Mahyar","doi":"10.1109/BELIV57783.2022.00008","DOIUrl":"https://doi.org/10.1109/BELIV57783.2022.00008","url":null,"abstract":"Appropriate evaluation and experimental design are fundamental for empirical sciences, particularly in data-driven fields. Due to the successes in computational modeling of languages, for instance, research outcomes are having an increasingly immediate impact on end users. As the gap in adoption by end users decreases, the need increases to ensure that tools and models developed by the research communities and practitioners are reliable, trustworthy, and supportive of the users in their goals. In this position paper, we focus on the issues of evaluating visual text analytics approaches. We take an interdisciplinary perspective from the visualization and natural language processing communities, as we argue that the design and validation of visual text analytics include concerns beyond computational or visual/interactive methods on their own. We identify four key groups of challenges for evaluating visual text analytics approaches (data ambiguity, experimental design, user trust, and \"big picture\" concerns) and provide suggestions for research opportunities from an interdisciplinary perspective.","PeriodicalId":299298,"journal":{"name":"2022 IEEE Evaluation and Beyond - Methodological Approaches for Visualization (BELIV)","volume":"57 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127050873","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Yasara Peiris, Clara-Maria Barth, Elaine M. Huang, J. Bernard
{"title":"A Data-Centric Methodology and Task Typology for Time-Stamped Event Sequences","authors":"Yasara Peiris, Clara-Maria Barth, Elaine M. Huang, J. Bernard","doi":"10.1109/BELIV57783.2022.00012","DOIUrl":"https://doi.org/10.1109/BELIV57783.2022.00012","url":null,"abstract":"Task abstractions and taxonomic structures for tasks are useful for designers of interactive data analysis approaches, serving as design targets and evaluation criteria alike. For individual data types, dataset-specific taxonomic structures capture unique data characteristics, while being generalizable across application domains. The creation of dataset-centric but domain-agnostic taxonomic structures is difficult, especially if best practices for a focused data type are still missing, observing experts is not feasible, and means for reflection and generalization are scarce. We discovered this need for methodological support when working with time-stamped event sequences, a datatype that has not yet been fully systematically studied in visualization research. To address this shortcoming, we present a methodology that enables researchers to abstract tasks and build dataset-centric taxonomic structures in five phases (data collection, coding, task categorization, task synthesis, and action-target-(criterion) crosscut). We validate the methodology by applying it to time-stamped event sequences and present a task typology that uses triples as a novel language of description for tasks: (1) action, (2) data target, and (3) data criterion. We further evaluate the descriptive power of the typology with a real-world case on cybersecurity.","PeriodicalId":299298,"journal":{"name":"2022 IEEE Evaluation and Beyond - Methodological Approaches for Visualization (BELIV)","volume":"7 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-09-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121108699","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Toward Inclusion and Accessibility in Visualization Research: Speculations on Challenges, Solution Strategies, and Calls for Action (Position Paper)","authors":"Katrin Angerbauer, M. Sedlmair","doi":"10.1109/BELIV57783.2022.00007","DOIUrl":"https://doi.org/10.1109/BELIV57783.2022.00007","url":null,"abstract":"Inclusion and accessibility in visualization research have gained increasing attention in recent years. However, many challenges still remain to be solved on the road toward a more inclusive, shared-experience-driven visualization design and evaluation process. In this position paper, we discuss challenges and speculate about potential solutions, based on related work, our own research, as well as personal experiences. The goal of this paper is to start discussions on the role of accessibility and inclusion in visualization design and evaluation.","PeriodicalId":299298,"journal":{"name":"2022 IEEE Evaluation and Beyond - Methodological Approaches for Visualization (BELIV)","volume":"97 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-09-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117318954","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}