Framework for tool evaluation for a maintenance environment

Tool evaluation forms an essential part of tool development. Unfortunately, there are only a few solutions for this activity. As part of the AMES project, tool evaluation was implemented as part of a more comprehensive framework, used with the development and application of several AMES tools for application maintenance. These included an application understanding tool-set, a disabbreviation tool, a reverse-engineering tool and an impact analysis tool-set. In addition, the requirement for applicability both to on-board space software and to tool software has directed our approach in setting requirements for tools evaluation. The central element of the evaluation process was an AMES-developed framework including evaluation criteria and the use of a goal/question/metrics (GQM) based approach. The criteria have produced detailed information about each tool. The final result can be regarded as a combination of the score for each criterion and explicit metrics data. In addition, descriptive information about applications was used in tool evaluation.  1998 John Wiley & Sons, Ltd.