On the Understandability of Semantic Constraints for Behavioral Software Architecture Compliance: A Controlled Experiment

Abstract

Software architecture compliance is concerned with the alignment of implementation with its desired architecture and detecting potential inconsistencies. The work presented in this paper is specifically concerned with behavioral architecture compliance. That is, the focus is on semantic alignment of implementation and architecture. In particular, this paper evaluates three representative approaches for describing semantic constraints in terms of their understandability, namely natural language descriptions as used in many architecture documentations today, a structured language based on specification patterns that abstract underlying temporal logic formulas, and a structured cause-effect language that is based on Complex Event Processing. We conducted a controlled experiment with 190 participants using a simple randomized design with one alternative per experimental unit. Overall all approaches support a high level of correct understanding, and the statistical inference suggests that all tested approaches are equally well suited for describing semantic constraints for behavioral architecture compliance in terms of understandability. In consequence this indicates that it is possible to benefit from the tested structured languages with underlying formal representations for automated verification without having to suffer from decreased understandability. Vice versa, the results suggest that the use of natural language can be a suitable way to document architecture semantics when reliable automated support for formal verification is of minor importance.