How to Evaluate Software Architectures: Tutorial on Practical Insights on Architecture Evaluation Projects with Industrial Customers

Abstract

Thorough and continuous architecting is the key to overall success in software engineering, and architecture evaluation is a crucial part of it. This tutorial presents a pragmatic architecture evaluation approach and insights gained from its application in more than 75 projects with industrial customers in the past decade. It presents context factors, empirical data, and example cases, as well as lessons learned on mitigating the risk of change through architecture evaluation. By providing comprehensive answers to many typical questions and discussing more frequent mistakes and lessons learned, the tutorial allows the audience to not only learn how to conduct architecture evaluations and interpret its results, but also to become aware of risks such as false conclusions, manipulating data, and unsound lines of argument. It equips the audience to become confident in assessing quantitative measurement results and recognize when it is better to rely on qualitative expertise. The target audience includes both practitioners and researchers. By demonstrating its impact and providing clear guidelines, data, and examples, it encourages practitioners to conduct architecture evaluations. At the same time, it offers researchers insights into industrial architecture evaluations, which serve as the basis for guiding research in this area and will inspire future research directions. Both groups will get an overview of the foundations and history of architecture evaluation. The tutorial covers the following important aspects of architecture evaluation • Architecture drivers: types of drivers, importance of drivers, elicitation of drivers, documentation of drivers as architecture scenarios • Solution Adequacy Check: check whether an architecture is adequate for its drivers, what are risks, assumptions, tradeoffs (based on ATAM (architecture tradeoff analysis method) and enhanced with other techniques for increasing confidence of results) • Documentation Quality Check: How adequate is the documentation of an architecture to be understandable and to serve its purposes? • Architecture Compliance Check: How to check whether an intended architecture is consistently reflected in the source code? For all aspects, pragmatic methodical support is provided in the tutorial and all checks are well integrated in the overall architecture evaluation method.