Designing the evaluation of operator-enabled interactive data exploration in VALIDE

Proceedings of the 2nd Workshop on Human-In-the-Loop Data Analytics. Workshop on Human-In-the-Loop Data Analytics (2nd : 2017 : Chicago, Ill.) Pub Date : 2022-06-12 DOI:10.1145/3546930.3547509

Yogendra Patil, S. Amer-Yahia, S. Subramanian

{"title":"Designing the evaluation of operator-enabled interactive data exploration in VALIDE","authors":"Yogendra Patil, S. Amer-Yahia, S. Subramanian","doi":"10.1145/3546930.3547509","DOIUrl":null,"url":null,"abstract":"Interactive Data Exploration (IDE) systems are technologies that facilitate the understanding of large datasets by providing high level easy-to-use operators. Compared to traditional querying systems, where users have to express each query, IDE systems allows users to perform expressive data exploration following the click-select-execute paradigm. Today, there exists no full-fledged evaluation framework for operator-enabled IDE. Most previous works are based on either logging user actions implicitly to compute quantitative metrics or running user studies to collect explicit feedback. Hence, there is a pressing need to articulate an evaluation framework that collects and compares quantitative human feedback along with system and data-centric evaluations. In this paper, we develop VALIDE, a preliminary design of a unified framework consisting of a methodology and metrics for IDE systems. VALIDE combines research from database benchmarking and human-computer interaction and will be demonstrated with a real IDE system.","PeriodicalId":92279,"journal":{"name":"Proceedings of the 2nd Workshop on Human-In-the-Loop Data Analytics. Workshop on Human-In-the-Loop Data Analytics (2nd : 2017 : Chicago, Ill.)","volume":"36 1","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2022-06-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"1","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Proceedings of the 2nd Workshop on Human-In-the-Loop Data Analytics. Workshop on Human-In-the-Loop Data Analytics (2nd : 2017 : Chicago, Ill.)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1145/3546930.3547509","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 1

Abstract

Interactive Data Exploration (IDE) systems are technologies that facilitate the understanding of large datasets by providing high level easy-to-use operators. Compared to traditional querying systems, where users have to express each query, IDE systems allows users to perform expressive data exploration following the click-select-execute paradigm. Today, there exists no full-fledged evaluation framework for operator-enabled IDE. Most previous works are based on either logging user actions implicitly to compute quantitative metrics or running user studies to collect explicit feedback. Hence, there is a pressing need to articulate an evaluation framework that collects and compares quantitative human feedback along with system and data-centric evaluations. In this paper, we develop VALIDE, a preliminary design of a unified framework consisting of a methodology and metrics for IDE systems. VALIDE combines research from database benchmarking and human-computer interaction and will be demonstrated with a real IDE system.

查看原文本刊更多论文

设计在validate中启用操作员的交互式数据探索的评估

交互式数据探索(IDE)系统是通过提供高层次的易于使用的操作符来促进对大型数据集的理解的技术。与传统查询系统(用户必须表达每个查询)相比，IDE系统允许用户按照单击-选择-执行范式执行表达性数据探索。目前，对于支持操作符的IDE，还没有成熟的评估框架。大多数先前的工作要么是基于隐式记录用户行为来计算定量指标，要么是基于运行用户研究来收集明确的反馈。因此，迫切需要明确一个评估框架，收集和比较定量的人类反馈以及以系统和数据为中心的评估。在本文中，我们开发了validide，这是一个由IDE系统的方法和度量组成的统一框架的初步设计。validate结合了数据库基准测试和人机交互的研究，并将在一个真实的IDE系统中进行演示。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Proceedings of the 2nd Workshop on Human-In-the-Loop Data Analytics. Workshop on Human-In-the-Loop Data Analytics (2nd : 2017 : Chicago, Ill.)

自引率

0.00%

发文量