Criteria for the evaluation of self-* systems

In the last years, the growing complexity of the current applications has led to the design of self-adapting systems presenting self-* properties. These systems are composed of several autonomous interactive entities. They behave autonomously and present enhanced characteristics allowing them to handle dynamics coming from exogenous and endogenous changes. In this paper, we propose a set of criteria for the description and evaluation of the adaptive properties of such systems. They aim to provide a concrete mechanism to analyze the quality of the design of adaptive systems, to evaluate the effect of self-* properties on the performances and to compare the adaptive features of different systems. The criteria are grouped into different categories: methodological, architectural, intrinsic, and runtime evaluation. They have been identified and specified by analyzing several case studies, which address self-adaptivity issues through different approaches with different objectives in various application contexts.