Scalable and Fault-Tolerant Cloud Computations: Modelling and Implementation

Abstract

This paper presents a formal model for science clouds, capable of predicting and controlling resources scalably, as well as its implementation as an open source solution, called Chiminey. The feasibility of Chiminey is shown using case studies on biophysics and structural chemistry computations. Big data is acquired from scientific instruments such as synchrotrons and atomic force microscopes. The model takes into account the architecture of the overall parallel and distributed system including large-scale data sources; data sinks, for example petabyte research data stores; and cluster or cloud virtual resources and infrastructures characterised by users in simple parameters upfront. Chiminey is developed to control large numbers of processes and to provide a reliable computing and data management, which can be used by researchers without having to learn extensive infrastructure concepts and technologies.