An exploratory empirical eye-tracker study of visualization techniques for coverage of combinatorial interaction testing in software product lines

IF 3.7 2区计算机科学 Q1 COMPUTER SCIENCE, SOFTWARE ENGINEERING

Journal of Systems and Software Pub Date : 2024-10-28 DOI:10.1016/j.jss.2024.112261

Kambiz Nezami Balouchi , Julien Mercier , Roberto E. Lopez-Herrejon

{"title":"An exploratory empirical eye-tracker study of visualization techniques for coverage of combinatorial interaction testing in software product lines","authors":"Kambiz Nezami Balouchi , Julien Mercier , Roberto E. Lopez-Herrejon","doi":"10.1016/j.jss.2024.112261","DOIUrl":null,"url":null,"abstract":"<div><div>Software Product Lines (SPLs) typically provide a large number of configurations to cater to a set of diverse requirements of specific markets. This large number of configurations renders unfeasible to test them all individually. Instead, Combinatorial Interaction Testing (CIT) computes a representative sample according to criteria of the interactions of features in the configurations. We performed an empirical study using eye-tracker technologies to analyze the effectiveness of two basic visualization techniques at conveying test coverage information of ten case studies of varying complexity. Our evaluation considered response accuracy, time-on-task, metacognitive monitoring, and visual attention. The study revealed clear advantages of a visualization technique over the other in three evaluation aspects, with a reverse effect depending on the strength of the coverage and distinct areas of visual attention.</div></div>","PeriodicalId":51099,"journal":{"name":"Journal of Systems and Software","volume":"220 ","pages":"Article 112261"},"PeriodicalIF":3.7000,"publicationDate":"2024-10-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Systems and Software","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0164121224003054","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, SOFTWARE ENGINEERING","Score":null,"Total":0}

引用次数: 0

Abstract

Software Product Lines (SPLs) typically provide a large number of configurations to cater to a set of diverse requirements of specific markets. This large number of configurations renders unfeasible to test them all individually. Instead, Combinatorial Interaction Testing (CIT) computes a representative sample according to criteria of the interactions of features in the configurations. We performed an empirical study using eye-tracker technologies to analyze the effectiveness of two basic visualization techniques at conveying test coverage information of ten case studies of varying complexity. Our evaluation considered response accuracy, time-on-task, metacognitive monitoring, and visual attention. The study revealed clear advantages of a visualization technique over the other in three evaluation aspects, with a reverse effect depending on the strength of the coverage and distinct areas of visual attention.

查看原文本刊更多论文

针对软件产品生产线组合交互测试覆盖范围的可视化技术的探索性实证眼动仪研究

软件产品系列 (SPL) 通常会提供大量配置，以满足特定市场的各种需求。由于配置数量庞大，对所有配置进行单独测试是不可行的。相反，组合交互测试（Combinatorial Interaction Testing，CIT）可以根据配置中特征的交互标准计算出具有代表性的样本。我们利用眼动仪技术进行了一项实证研究，分析了两种基本可视化技术在传达十个不同复杂程度的案例研究的测试覆盖信息方面的有效性。我们的评估考虑了反应准确性、任务时间、元认知监控和视觉注意力。研究表明，在三个评估方面，一种可视化技术比另一种技术具有明显优势，而根据覆盖范围的强弱和视觉注意力的不同区域，效果则相反。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Journal of Systems and Software 工程技术-计算机：理论方法

CiteScore

8.60

自引率

5.70%

发文量

193

审稿时长

16 weeks

期刊介绍： The Journal of Systems and Software publishes papers covering all aspects of software engineering and related hardware-software-systems issues. All articles should include a validation of the idea presented, e.g. through case studies, experiments, or systematic comparisons with other approaches already in practice. Topics of interest include, but are not limited to: • Methods and tools for, and empirical studies on, software requirements, design, architecture, verification and validation, maintenance and evolution • Agile, model-driven, service-oriented, open source and global software development • Approaches for mobile, multiprocessing, real-time, distributed, cloud-based, dependable and virtualized systems • Human factors and management concerns of software development • Data management and big data issues of software systems • Metrics and evaluation, data mining of software development resources • Business and economic aspects of software development processes The journal welcomes state-of-the-art surveys and reports of practical experience for all of these topics.