Proceedings of the 2nd International Workshop on Practical Reproducible Evaluation of Computer Systems最新文献

Provenance and Reproducibility in the Automation of a Standard Computational Neuroscience Pipeline 标准计算神经科学流水线自动化的来源和再现性

Proceedings of the 2nd International Workshop on Practical Reproducible Evaluation of Computer Systems Pub Date : 2019-06-17 DOI: 10.1145/3322790.3330592

David B. Stockton, A. Prinz, F. Santamaría

引用次数: 0

Scientific Tests and Continuous Integration Strategies to Enhance Reproducibility in the Scientific Software Context 在科学软件环境中提高可重复性的科学测试和持续集成策略

Proceedings of the 2nd International Workshop on Practical Reproducible Evaluation of Computer Systems Pub Date : 2019-06-17 DOI: 10.1145/3322790.3330595

M. Krafczyk, A. Shi, A. Bhaskar, D. Marinov, V. Stodden

{"title":"Scientific Tests and Continuous Integration Strategies to Enhance Reproducibility in the Scientific Software Context","authors":"M. Krafczyk, A. Shi, A. Bhaskar, D. Marinov, V. Stodden","doi":"10.1145/3322790.3330595","DOIUrl":"https://doi.org/10.1145/3322790.3330595","url":null,"abstract":"Continuous integration (CI) is a well-established technique in commercial and open-source software projects, although not routinely used in scientific publishing. In the scientific software context, CI can serve two functions to increase reproducibility of scientific results: providing an established platform for testing the reproducibility of these results, and demonstrating to other scientists how the code and data generate the published results. We explore scientific software testing and CI strategies using two articles published in the areas of applied mathematics and computational physics. We discuss lessons learned from reproducing these articles as well as examine and discuss existing tests. We introduce the notion of a \"scientific test\" as one that produces computational results from a published article. We then consider full result reproduction within a CI environment. If authors find their work too time or resource intensive to easily adapt to a CI context, we recommend the inclusion of results from reduced versions of their work (e.g., run at lower resolution, with shorter time scales, with smaller data sets) alongside their primary results within their article. While these smaller versions may be less interesting scientifically, they can serve to verify that published code and data are working properly. We demonstrate such reduction tests on the two articles studied.","PeriodicalId":192842,"journal":{"name":"Proceedings of the 2nd International Workshop on Practical Reproducible Evaluation of Computer Systems","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-06-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130379731","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 23

Implementing Computational Reproducibility in the Whole Tale Environment 在整个故事环境中实现计算再现性

Proceedings of the 2nd International Workshop on Practical Reproducible Evaluation of Computer Systems Pub Date : 2019-06-17 DOI: 10.1145/3322790.3330594

K. Chard, N. Gaffney, Matthew B. Jones, K. Kowalik, Bertram Ludäscher, J. Nabrzyski, V. Stodden, I. Taylor, M. Turk, C. Willis

引用次数: 12

Initial Thoughts on Cybersecurity And Reproducibility 关于网络安全和可重复性的初步思考

Proceedings of the 2nd International Workshop on Practical Reproducible Evaluation of Computer Systems Pub Date : 2019-06-17 DOI: 10.1145/3322790.3330593

E. Deelman, V. Stodden, M. Taufer, Von Welch

引用次数: 5

Reproducible Computer Network Experiments: A Case Study Using Popper 可重复计算机网络实验:以Popper为例

Proceedings of the 2nd International Workshop on Practical Reproducible Evaluation of Computer Systems Pub Date : 2019-06-17 DOI: 10.1145/3322790.3330596

Andrea David, Mariette Souppe, I. Jimenez, K. Obraczka, Sam Mansfield, K. Veenstra, C. Maltzahn

{"title":"Reproducible Computer Network Experiments: A Case Study Using Popper","authors":"Andrea David, Mariette Souppe, I. Jimenez, K. Obraczka, Sam Mansfield, K. Veenstra, C. Maltzahn","doi":"10.1145/3322790.3330596","DOIUrl":"https://doi.org/10.1145/3322790.3330596","url":null,"abstract":"Computer network research experiments can be broadly grouped in three categories: simulated, controlled, and real-world experiments. Simulation frameworks, experiment testbeds and measurement tools, respectively, are commonly used as the platforms for carrying out network experiments. In many cases, given the nature of computer network experiments, properly configuring these platforms is a complex and time-consuming task, which makes replicating and validating research results quite challenging. This complexity can be reduced by leveraging tools that enable experiment reproducibility. In this paper, we show how a recently proposed reproducibility tool called Popper facilitates the reproduction of networking experiments. In particular, we detail the steps taken to reproduce results in two published articles that rely on simulations. The outcome of this exercise is a generic workflow for carrying out network simulation experiments. In addition, we briefly present two additional Popper workflows for running experiments on controlled testbeds, as well as studies that gather real-world metrics (all code is publicly available on Github). We close by providing a list of lessons we learned throughout this process.","PeriodicalId":192842,"journal":{"name":"Proceedings of the 2nd International Workshop on Practical Reproducible Evaluation of Computer Systems","volume":"20 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-06-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116178321","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2

Applicability Study of the PRIMAD Model to LIGO Gravitational Wave Search Workflows PRIMAD模型在LIGO引力波搜索工作流程中的适用性研究

Proceedings of the 2nd International Workshop on Practical Reproducible Evaluation of Computer Systems Pub Date : 2019-04-10 DOI: 10.1145/3322790.3330591

Dylan Chapp, Danny Rorabaugh, Duncan A. Brown, E. Deelman, K. Vahi, Von Welch, M. Taufer

引用次数: 2

Proceedings of the 2nd International Workshop on Practical Reproducible Evaluation of Computer Systems 第二届计算机系统实用可重复评估国际研讨会论文集

Proceedings of the 2nd International Workshop on Practical Reproducible Evaluation of Computer Systems Pub Date : 1900-01-01 DOI: 10.1145/3322790

引用次数: 2