Design Science Research in Practice: What Can We Learn from a Longitudinal Analysis of the Development of Published Artifacts?

Abstract

Aim/Purpose: To discuss the Design Science Research approach by comparing some of its canons with observed practices in projects in which it is applied, in order to understand and structure it better. Background: Recent criticisms of the application of the Design Science Research (DSR) approach have pointed out the need to make it more approachable and less confusing to overcome deficiencies such as the unrealistic evaluation. Methodology: We identified and analyzed 92 articles that presented artifacts developed from DSR projects and another 60 articles with preceding or subsequent actions associated with these 92 projects. We applied the content analysis technique to these 152 articles, enabling the preparation of network diagrams and an analysis of the longitudinal evolution of these projects in terms of activities performed and the types of artifacts involved. Contribution: The content analysis of these 152 articles enabled the preparation of network diagrams and an analysis of the longitudinal evolution of these projects in terms of the activities and types of artifacts involved. Evidence was found of a precedence hierarchy among different types of artifacts, as well as nine new opportunities for entry points for the continuity of DSR studies. Only 14% of the DSR artifacts underwent an evaluation by typical end users, characterizing a tenth type of entry point. Regarding the evaluation process, four aspects were identified, which demonstrated that 86% of DSR artifact evaluations are unrealistic. Findings: We identified and defined a set of attributes that allows a better characterization and structuring of the artifact evaluation process. Analyzing the field data, we inferred a precedence hierarchy for different artifacts types, as well as nine new opportunities for entry points for the continuity of DSR studies. Recommendation for Researchers: The four attributes identified for analyzing evaluation processes serve as guidelines for practitioners and researchers to achieve a realistic evaluation of artifacts. Future Research: The nine new entry points identified serve as an inspiration for researchers to give continuity to DSR projects.