Experience with Performance Engineering Training in Academic and Industrial Environments

Abstract

We describe our experience of teaching software performance engineering and analysis in academic and industrial settings. There is a need to teach course participants core principles about basic performance modeling and data handling in both settings. These principles are independent of the technologies to which they must be applied and of the measurement and load generation tools that support them. Our experience suggests that it is more difficult for course participants in a university to relate performance engineering to such aspects of the software life cycle as architecture and requirements engineering than it is for industrial participants, because university students are likely lack the related development experience. This is especially true in countries where it is uncommon for mature students to return to graduate school after working outside a university. Our experience also suggests that students who are trained in the natural sciences or who have been rigorously trained in statistics, operations research, or industrial engineering are more likely to devise sound performance tests and be more comfortable with manipulating, interpreting, and analyzing performance data than computer science majors who do not have this type of training. On the other hand, industrial course participants may have had experience of requirements engineering and functional testing processes, but not be used to performing quantitative analyses or devising performance tests to be informative about system behavior and scalability. Some of them may also have considerable experience of using measurement tools and load generators, though not always to the best advantage. Our goal in performance training is to show participants the relationship between performance requirements engineering, testing, architecture, and modeling in principle and in practice. When all participants are involved in the same project in a company, this could be achieved by linking training to a project's performance needs. Otherwise, this can be achieved with exercises that include measuring the resource usage of various applications on their own laptops and explaining their observations. In either case, mentoring of practitioners should include stimulating their curiosity to understand new technologies and to applying various optimization and analysis methods to the issues they encounter.