Improving CSE software through reproducibility requirements

Abstract

It is often observed that software engineering (SE) processes and practices for computational science and engineering (CSE) lag behind other SE areas [7]. This issue has been a concern for funding agencies, since new research increasingly relies upon and produces computational tools. At the same time, CSE research organizations find it difficult to prescribe formal SE practices for funded projects. Theoretical and experimental science rely heavily on independent verification of results as part of the scientific process. Computational science should have the same regard for independent verification but it does not. In this paper, we present an argument for using reproducibility and independent verification requirements as a driver to improve SE processes and practices. We describe existing efforts that support our argument, how these requirements can impact SE, challenges we face, and new opportunities for using reproducibility requirements as a driver for higher quality CSE software.