Isn’t it about time to meet DORA?

Abstract

Given that DORA is rarely mentioned in scholarly journals devoted to the information systems (IS) field, it is likely safe to say that most contributors to, and users of, this and other IS journals are unfamiliar with DORA. Although the same holds for other functional areas of business, the situation is quite different in scientific and medical disciplines. Just maybe, these disciplines are onto something – something that could benefit ongoing development and recognition of IS as a discipline. So, what is DORA? Should we care? Why? The answers are important for shaping the development/future of IS as a scholarly discipline, encouraging/facilitating innovation in IS research, allowing/fostering research liberty for young IS scholars, and spurring a productive academic career that avoids disillusionment. Before addressing the questions, think about the state of approaches to evaluating researchers for merit, promotion, tenure, or funding purposes. Disregarding intrusions of organizational politics and personal relationships (which can be quite vexing to those on the short end of power), it seems typical to focus on where a researcher has published his or her work. It is most common for evaluation to focus on placement of articles in journals. This is usually the case for IS, other business disciplines, physical sciences, live sciences, and medical fields. Simply put, the value of a publication is considered to depend primarily on the journal in which it appears. In other words, the value of a gift is considered to depend on judgments about the wrapping that contains it, rather than the nature or utility of the gift itself (Zhang, Rousseau, and Sivertsen 2017). If it has a particular wrapping, then it must certainly be of the highest value. It follows that, if packaged in a different type of wrapping, that very same gift must be less valuable. Adopting such a method for evaluating a researcher’s work means that an article is judged to have less/more merit if it appears in one journal rather than another. Being in one journal versus another somehow diminishes/improves an article’s merit. The article is perceived as becoming imbued with a halo that a journal exudes, and that halo is seen as defining the article’s value. Adopting the halo method of research evaluation transforms the problem from one of assessing the merit of individual articles to the task of assessing the merit of individual journals (i.e., the strengths of their halos; the shininess/aesthetics/approval signaled by the wrapping papers). To support the efforts of evaluators, various approaches have been advanced/adopted for settling on the relative merits of journals in a field. Ultimately, it is the evaluator (or evaluator’s superiors) who selects what journal evaluation approach will be used in a halo exercise. Approaches range from largely subjective to largely objective. Comparative examples of various evaluation approaches can be found in a series of quizzes that are applied to the context of IS journals (Chen and Holsapple 2013). In the subjective case, the degree of strength attributed to a journal’s halo is determined by the evaluator’s (superior’s) vantage point, perspective, interpretations, preconceptions, training, biases, values, and so forth. Examples include relying on tradition, accepting pronouncements by others, or tailoring that accounts for journals that specialize in topics of particular emphasis by an institution or funding agency. In an effort to mitigate drawbacks of subjective methods, more data-driven methods to assess journal merit have been devised, applying various techniques to various kinds of data sets in order to produce: ● a numeric rating for each journal of interest, reflecting the strength of its halo ● a classification of journals into tiers, where the journals in a tier have comparable halos, but differ notably from those in other tiers. JOURNAL OF ORGANIZATIONAL COMPUTING AND ELECTRONIC COMMERCE 2018, VOL. 28, NO. 4, 287–290 https://doi.org/10.1080/10919392.2018.1522774