Fit, scope and the shifting baseline: Is your submission likely to be desk rejected?

Abstract

The field of information systems (IS) has long had to navigate challenges in defining its scope, often straddling the realms of computer science, psychology, management and organisational behaviour. This positioning has rendered its boundaries occasionally ambiguous (Sarker et al., 2019), yet at the same time the field has remained undeniably important in its relevance to practice.

This ongoing debate about the field's boundaries and identity has persisted over several decades (Alavi & Carlson, 1992; Benbasat & Zmud, 2003; Palvia et al., 2004; Sarker et al., 2019) and has followed both technological advancements and methodological innovations. The challenge of encapsulating the breadth of IS is illustrated by early attempts to categorise the discipline through keyword schemes (Barki et al., 1993), which, even though encompassing nearly 1300 keywords, still fall short of covering the full range of topics that IS journals publish today (Upreti et al., 2023). Indeed, defining the intellectual core of the IS discipline (Sidorova et al., 2008) is a challenging task (Chatterjee et al., 2021; Schwartz, 2014).

Regardless of the varying opinions on what IS is (or is not) and thus what the IS discipline entails, some basic tenets of the IS discipline have become broadly accepted over time. Most scholars would agree that: IS is a sociotechnical discipline where social and technical factors acquire comparative salience in terms of their relationships with outcomes (Beath et al., 2013; Sarker et al., 2019); IS research needs theoretical grounding (Díaz Andrade et al., 2023); IS scholarship has substantial diversity (Taylor et al., 2010); and IS research benefits from both intra- and inter-disciplinary approaches (Tarafdar & Davison, 2018).

Nevertheless, we feel that these well-accepted characteristics of IS research sometimes conflict with each other, creating confusion in the minds of authors submitting their work to IS journals. For example, the interdisciplinary and diverse nature of the IS research landscape often prompts authors from other disciplines (notably engineering, business and the social sciences) to submit their work to IS journals without reflecting upon whether the work is truly sociotechnical and has an identifiable IS artefact (Chatterjee et al., 2021) or even if it has a strong theoretical grounding (Davison & Tarafdar, 2018) that is consistent with how the IS discipline understands theory (Gregor, 2006).

ISJ is also encountering this trend: we often see situations involving authors who, in their submitted papers, make no connection to any of the fundamental tenets of IS research, such as a carefully-articulated theoretical contribution, or an identifiable sociotechnical perspective. We particularly observe that over the past 5 years, there has been a significant shift in the focus of the papers we receive at ISJ, evolving from econometrics and analytics towards more predominantly technical solutions using natural language processing, machine learning and artificial intelligence. These papers are often largely technical (Sarker et al., 2019), and the contribution that they offer in terms of expanding the frontiers of IS knowledge is often minimal or even non-existent. Thus, we discern that the vast majority of these papers have a very tenuous connection to IS, as it is broadly understood by the scholarly practitioners of the discipline.

A quick perusal of the references cited in these kinds of papers reveals large numbers of IEEE Transactions, some specialised computer science journals and a similar assortment of curious animals, viz.: slime mould, grey wolf, seagull, rat, pelican, …. We definitively infer that such papers are not likely to be the primary interest to a scholarly or practice-oriented IS audience.

In addition, other common features of these inappropriate submissions are worth highlighting: keywords (which are often in the title) such as big data, algorithm, blockchain, deep learning, etc.; pages of mathematical formulae; and a complete absence of reference to work published in IS journals. Naturally, some papers published in IS journals are legitimately concerned with these same keywords, and some might also have pages of mathematical formulae. Perhaps, authors are inspired to imagine that ISJ (or other IS journals) are appropriate outlets for such work. But it is quite normal, in any discipline, that authors should cite research already published in that discipline. This is not an exclusive requirement (i.e., it is acceptable to cite articles published in other disciplines), but it seems bizarre to read a paper that does not cite even one article published in the discipline of the journal.

Another common feature of these submissions is the complete absence of theory. As we have recently written in an editorial in this journal (Díaz Andrade et al., 2023), it is mandatory for Research Articles to make a contribution to theory. This is non-negotiable! This is not to say that we never publish theory-free papers: some submission types do not (need to) make theory contributions, notably research methods papers, opinion papers and practitioner papers. But the sheer number of inappropriate submissions is bewildering.

Why do all these authors, many of whom work in Engineering faculties and whose Google Scholar profiles exhibit no prior evidence of ever publishing in any IS journal, submit to the ISJ? Upon discussing with fellow ISJ SEs and AEs, coupled with our own experience, we discerned some plausible underlying reasons.

First, over the past decade, business school-based IS departments have increasingly recruited scholars with specialised technical skills to instruct courses in business analytics, ML and AI. These scholars often have heavily technical backgrounds, which is reflected in their research. However, upon joining business schools, they encounter the challenge of aligning their work with business school journal rankings, such as the Senior Scholars' List of Premier Journals, the Chartered Association of Business Schools (CABS) in the UK, the Australian Business Deans Council (ABDC) or the Financial Times Top 50 (FT50). This requirement presents a significant dilemma, compelling them to tailor their research to fit the IS discipline more closely, or to imagine that their predominantly technical work will be acceptable to an IS audience. This may explain the noticeable increase in the volume of technically-oriented submissions that lack a clear IS anchor.

Second, there are increasing numbers of PhD students in IS departments who are investigating the kinds of topics that are reflected in the keywords above. These students have to submit somewhere, and IS journals seem to be an obvious choice. After all, they are IS students! It is also true that IS, compared with other business disciplines, apportions considerable salience to the role of technology in shaping societies and organisations, and is probably the most ‘technical’ field within the typical business school. Many of these IS PhD students and junior scholars have backgrounds in engineering and computer science (especially in countries and education systems that predominantly value a strong STEM training), and their submissions tend to reflect their technical skills.

A third possibility is the nature of the IS field itself, and its diversity and inherent interdisciplinarity. IS scholars can reside within business schools (this is most prominent), but they can also reside within information schools, and even schools of mathematics and computing, or the social sciences. Naturally, if an IS scholar resides within other units than business schools, and works with authors in those schools, they can collaborate to develop work that has greater degrees of separation from what IS research is commonly understood to be, particularly in business schools.

But why do these submissions a lack of theory? This may be due to recent trends in IS research: increasingly theory seems to be seen as a quaint relic of the past but no longer a de rigueur component of contemporary research (Hirschheim, 2019). The availability of large volumes of secondary data seems to have exacerbated this trend, where sophisticated empirical exercises, not necessarily accompanied by theoretical vision or solid theory development, seem to be acceptable to some reviewers and editors as legitimate forms of research. The rise of analytics and data mining has prompted work, which would have been criticised (perhaps 10–15 years ago) for being atheoretical or largely exploratory and simply pattern-seeking. However, this work is now welcomed, and together with the increase in the number of technical experts employed in business schools, may be further contributing towards this trend. In short, the perception of what is considered legitimate IS research may be changing. This is thus a shifting baseline (Davison & Tarafdar, 2018): it is not to say that theoretical contributions are no longer valued but rather to say that the baseline, at last for some authors/reviewers/journals, has now extended to include atheoretical work.

While these possibilities are relevant for all IS journals, our discussions with editors of other journals seem to suggest that only a handful of journals in the Senior Scholars' List of Premier Journals are significantly afflicted by these issues. That led us to question whether there is something unique about the ISJ that prompts the current situation. For example, given that many of these submitting authors are affiliated with engineering and technological departments, is it the case that ISJ may be perceived as being softer or easier to publish in, compared with a computer science outlet, especially an IEEE journal?1 The ISJ may also be attractive because of its relatively high impact factor. Alternatively, do any of the ISJ's current senior and associate editors work in areas that might signal that the ISJ is open to accepting such submissions? Similarly, has the ISJ already published articles with similar keywords to these inappropriate submissions? Finally, is there something in the professed scope and vision in ISJ (perhaps on the website) that inadvertently misleads the authors into thinking that ISJ is a good fit for such types of research?

A variety of means can be used to draw the attention of potential authors to matters that are critically related to their submissions, and the likelihood that these submissions are out of scope for a journal. Unfortunately, there is no way to guarantee that authors will care about this information, read it even if required to do so, or act on it. A very strict front-end screening and desk reject policy can weed out inappropriate submissions, which has the merit of protecting the time and energy of the senior and associate editors, as well as reviewers. However, we do not counsel that front-end screening be automated with AI. Despite the amount of time and energy that a human screening editor may need to expend, it is critical that we do not desk reject a submission by mistake. Indeed, as Davison (2024) argues, all authors, even those whose behaviour leads much to be desired or whose submissions are completely out of scope, deserve to be treated with respect and patience.

We note that Wiley currently operates an AI-based journal selection program that takes as input the title and abstract, and delivers as output a list of journals that might be suitable, together with a relevance indicator for each one.2 A similar approach could be taken to provide authors with an AI-based fit score after the meta data (title and abstract) have been provided in the submission system but before the authors complete their submission. In this way, authors could be informed about the likely fit or misfit of their paper for the specific journal. Notwithstanding these AI-based approaches, we caution authors to consider most carefully whether the journal they are submitting to is indeed an appropriate outlet for their research.