A benchmarking method for information systems

Abstract

Benchmark tests are used to evaluate the performance of information systems and to test their compliance with user requirements. In early project phases benchmark tests help to make build-or-buy decisions and to estimate the acceptance and performance an information system can achieve in a given environment. For a benchmark test, a team of future users visits candidate software vendors and executes pre-defined scenarios on their software systems using dedicated test data. Test scenarios and data are created using business-process modeling with UML. During the test, each team member observes and rates the system performance. For customers, benchmark results are powerful tools for project management, while vendors gain valuable insight into requirements and established practices of their target customers.