A methodology for reproducible and portable experiment workflows

IF 4.5 3区计算机科学 Q1 COMPUTER SCIENCE, INFORMATION SYSTEMS

Computer Communications Pub Date : 2025-04-29 DOI:10.1016/j.comcom.2025.108178

Henning Stubbe, Sebastian Gallenmüller, Georg Carle

{"title":"A methodology for reproducible and portable experiment workflows","authors":"Henning Stubbe, Sebastian Gallenmüller, Georg Carle","doi":"10.1016/j.comcom.2025.108178","DOIUrl":null,"url":null,"abstract":"<div><div>Testbeds allow the creation of research prototypes to test new ideas through practical experiments. This central role in validating ideas makes them irreplaceable tools for data-driven research in computer science. Various testbeds were created to provide testbeds for the scientific community. To simplify testbed usage, frameworks help to authenticate users, allocate resources, and run experiments. Each testbed typically implements its own framework using a specific API to realize experiments. Such an experiment design impedes the portability of experiments between different testbeds. In this paper, we present a solution where we port the pos experiment controller to the Chameleon and CloudLab testbed. The well-structured pos experiment workflow allows the creation of inherently reproducible experiments. Previously, the experiments using the pos workflow were only possible in dedicated testbeds. By introducing the portability feature, these experiments can run on Chameleon and CloudLab. We demonstrate that experiments can be executed on any of the mentioned platforms without changing the experiment definition. Based on these results, we discuss how the portability feature will be used in the upcoming SLICES-RI testbeds to create reproducible and easily-shareable experiments.</div></div>","PeriodicalId":55224,"journal":{"name":"Computer Communications","volume":"239 ","pages":"Article 108178"},"PeriodicalIF":4.5000,"publicationDate":"2025-04-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Computer Communications","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0140366425001355","RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}

引用次数: 0

Abstract

Testbeds allow the creation of research prototypes to test new ideas through practical experiments. This central role in validating ideas makes them irreplaceable tools for data-driven research in computer science. Various testbeds were created to provide testbeds for the scientific community. To simplify testbed usage, frameworks help to authenticate users, allocate resources, and run experiments. Each testbed typically implements its own framework using a specific API to realize experiments. Such an experiment design impedes the portability of experiments between different testbeds. In this paper, we present a solution where we port the pos experiment controller to the Chameleon and CloudLab testbed. The well-structured pos experiment workflow allows the creation of inherently reproducible experiments. Previously, the experiments using the pos workflow were only possible in dedicated testbeds. By introducing the portability feature, these experiments can run on Chameleon and CloudLab. We demonstrate that experiments can be executed on any of the mentioned platforms without changing the experiment definition. Based on these results, we discuss how the portability feature will be used in the upcoming SLICES-RI testbeds to create reproducible and easily-shareable experiments.

查看原文本刊更多论文

一种可重复和便携式实验工作流程的方法

试验台允许创建研究原型，通过实际实验来测试新想法。在验证思想方面的核心作用使它们成为计算机科学中数据驱动研究的不可替代的工具。各种各样的试验台被创造出来，为科学界提供试验台。为了简化测试平台的使用，框架可以帮助验证用户、分配资源和运行实验。每个测试平台通常使用特定的API实现自己的框架来实现实验。这样的实验设计阻碍了不同试验台之间实验的可移植性。在本文中，我们提出了一个解决方案，我们将pos实验控制器移植到变色龙和CloudLab测试平台上。结构良好的pos实验工作流允许创建固有的可重复实验。以前，使用pos工作流的实验只能在专用的测试平台上进行。通过引入可移植性特性，这些实验可以在Chameleon和CloudLab上运行。我们证明实验可以在任何上述平台上执行，而无需改变实验定义。基于这些结果，我们讨论了如何在即将推出的SLICES-RI测试平台中使用可移植性特性来创建可重复且易于共享的实验。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Computer Communications 工程技术-电信学

CiteScore

14.10

自引率

5.00%

发文量

397

审稿时长

66 days

期刊介绍： Computer and Communications networks are key infrastructures of the information society with high socio-economic value as they contribute to the correct operations of many critical services (from healthcare to finance and transportation). Internet is the core of today''s computer-communication infrastructures. This has transformed the Internet, from a robust network for data transfer between computers, to a global, content-rich, communication and information system where contents are increasingly generated by the users, and distributed according to human social relations. Next-generation network technologies, architectures and protocols are therefore required to overcome the limitations of the legacy Internet and add new capabilities and services. The future Internet should be ubiquitous, secure, resilient, and closer to human communication paradigms. Computer Communications is a peer-reviewed international journal that publishes high-quality scientific articles (both theory and practice) and survey papers covering all aspects of future computer communication networks (on all layers, except the physical layer), with a special attention to the evolution of the Internet architecture, protocols, services, and applications.