Runtime generation of data processors on local user computers

Abstract

Data interoperability in scientific research is a major challenge. Heterogeneous data file formats, data structure formats, and data storage schemas are prevalent in the scientific research community. This poses a great challenge for collaboration between researchers due to data interoperability issues. In our prior work, we have addressed these challenges by proposing a web-enabled approach for generating data processors initially intended for environmental science researchers. This paper takes the solution one step further, and describes the software that enables generating and running data processors on local user computers. The paper details the design of the software solution and presents the steps that the users need to follow for running processors for a variety of data conversion needs. The solution presented is powerful and flexible as it can be applied to a large variety of data conversion needs and to practically any type of data-intensive scientific research. It also facilitates collaboration as scientists can share their custom created data processors, thus increasing efficiency in research activities. Researchers can benefit from the software engineering principles used in creating and generating data processors, and can take advantage of the reusability of numerous data processors create by end users who do not necessarily have software development expertise.