Reviews, recommendations and decisions: Contrasting perspectives

As a junior scholar, I used to be petrified of reviewers. Perhaps you are too. Reviewers, it appeared, were the ultimate arbiters of whether my paper could be published. Consequently, satisfying the reviewers became the be-all and end-all of the publication process. This fear of reviewers is not limited to junior scholars. As Allen Lee (cited in Davison, 2019) noted: ‘I believe that many or most seasoned researchers write with reviewers in mind, if not for the first submission, then certainly by the third revision. In fact, the exasperation is so great by the third or fourth revision that the general audience has completely disappeared from the author's mind and the author is left just wanting to address what the reviewers and editor want’. Even now, as I cross the 30-year mark as an academic who is both an editor and an author, reviewers can exert a baneful influence.

Although this is an editorial, I am conscious that I am writing it from both editorial and authorial perspectives. As an author, I am shielded by tenure to some degree, but my dean still hopes that I will publish in what he deems to be the top journals in the field (I may not agree about that) and has the carrots to induce compliance. Deans apart, I also enjoy conducting research and the writing process, so I can say that I write for my own pleasure. However, my papers also witness the sharp end of reviewers' tongues (or fingers) with comments that can be disheartening, destructive or ridiculous. What should I make of the reviewer who suggested that in future I should not bother with ethnographies because they are too subjective and instead collect survey data? Or the reviewer who advised that collecting data in China is of little value because it cannot be generalised to anywhere else, whereas data collected in the USA is universally generalisable. Honestly, I am not making this up; I could not! These perspectives are so far beyond the pale of reasonable human thinking as to be in the category of ‘truth is stranger than fiction’.

Meanwhile, as editor, I see the best and the worst of reviewers (and of course everything in between). Reviewers can be polite and constructive to a fault, but they can also be vicious, self-serving and/or just plain nasty, seeming to revel in their power to determine the fate of a manuscript. As editor, can I ‘edit’ the reviews before I send them to authors? Strictly speaking, this depends on publisher policy. At some journals, such editing is allowed, whereas at others it is not. I know one editor who routinely edits all reviewer comments for style and readability. But there are many editors who don't. Personally, I try not to micromanage the process unless it is really critical. Reviewers who are offensive will find their reviews rescinded, that is, returned with a request that they moderate their language or tone; some comply, others do not.

Nevertheless, conversations with my colleagues have reinforced that sense that reviewers must (somehow) be satisfied. I suspect that anyone who has been in the position of reviewer (which means most of us) knows this and, paradoxically, is happy to exert whatever power they can conjure up. The apparently split personality of the fearful author|fear-inducing reviewer is both bizarre and common, but then humans are well known for their behavioural differences when in juxtaposed positions, notably where power dynamics are inverted.

Some years ago, a former editor in chief of MISQ remarked to me that he no longer enjoys academic paper writing for the simple reason that evil reviewers have personality disorders and are out to get you. Does that sound a bit exaggerated? Well, it probably is and since neither of us is a qualified psychiatrist, we are perhaps using ‘personality disorder’ and ‘split personality’ rather loosely, but there is a kernel of truth in here, nonetheless. Unfortunately, there are occasions when reviewers have more power than should be the case. This is generally what happens when an editor is not doing their job properly. I have been on the receiving end of editor decisions where the editor simply writes ‘all the reviewers recommend rejection, so the paper is rejected’. The editor's opinion (if any) is not communicated at all. So yes, here the reviewers, at least collectively, seem to have a lot of power. Even when editors are prepared to share their thoughts, editors can also go along with the majority of reviewers and be unwilling to take the trouble to formulate their own opinion.

Here is an example. Two reviews are submitted for a paper. One reviewer argues for major revisions (but in a private note explains that there is a huge amount of work to do, and the risk level is high), while the other argues for rejection. The Associate Editor (AE) feels torn. He would actually prefer to give the authors a chance to revise, thinking that the reviewer comments, although valid, are quite easily addressable and not as fatal or dramatic as the reviewers make out. The AE writes to the Senior Editor (SE) ‘What should I do? Am I allowed to make a recommendation that is manifestly different from the perspective of the authors?’. The answer to that question is a resounding YES. It is not the AE's job simply to find reviewers and then to communicate their recommendation. Instead, the AE has the responsibility to form his/her own opinion about the manuscript. That opinion may be informed by the reviewers, but it should never be determined by the reviewers. The same goes for the SE. As Editor in Chief, I actually don't expect to see a perfectly unified set of opinions across the review team of 4–5 people. It can happen, but it is unusual. More usually, the review team is in some way split. Each person exhibits some degree of appreciation, critical opinion, positive attitude, negativity, and so on, and often inconsistently.

Now, let us say that as Editor I send the authors a Revise and Resubmit decision. Yes, there is work to do and it is not easy, but it is doable. Reviewer 1 will be easier to satisfy, and Reviewer 2 will require more work. But then the author of the paper writes to me as follows: ‘I am not certain that I will be able to satisfy the concerns of R2. I will read the comments again in detail and give it a lot of thought. If I don't think I can, I will just withdraw the paper, so not to waste anyone's time’. My reaction is ‘Please don't do that! I don't see Reviewer 2 as a bar to publication. One reviewer is not making the decision. I have no problem accepting a paper that one reviewer doesn't like. Please do your best, and feel free to push back on Reviewer 2's comments as necessary. If you want to have a conversation about this, please reach out to me’ (cf. Techatassanasoontorn & Davison, 2022). Indeed, this situation reminds me of another where the SE of one of our premier journals rejected a paper in the fifth round because one reviewer could not or would not be satisfied. The rest of the review team (SE, AE, 2 other reviewers) was satisfied, but this one person held out for more major changes. The authors refused to make the changes, and the editor rejected the paper. That is an appalling situation, demonstrating not only a complete lack of intelligence, but also a complete lack of willingness to exercise editorial prerogative and make a decision that reflects the SE's view. The author approached me, forlornly, asking for advice. I looked at the paper and could see nothing wrong with it at all, bar a few small typos. I invited the author to submit it to the ISJ, which he did, and it was accepted, following minor revisions, within a week. In the year following publication, it became the most highly cited paper of the year, and the citations still come in. Over a five-year period, it received close to 1000 Google Scholar citations. But one reviewer seemed determined to frustrate that outcome Why?

Although I have written about the SE and AE roles elsewhere (Tarafdar & Davison, 2021), the point is worth reiterating. I do expect that SEs and AEs will do their jobs properly. They are not there just to recruit reviewers and then treat the reviewer recommendations as votes. Instead, they should exert some effort, read the papers assigned to them, and write up their own independent analyses. Clearly some do a better job of this than others. Why might they not want to take the trouble? One reason, especially for AEs, may be that they are not senior enough and do not want to make more senior reviewers angry by ignoring their recommendations. This alone can be a good reason for not appointing junior scholars as AEs. I find that these days people are very eager to become AEs and SEs, but the ‘duty of care’ and ‘attention to quality’ are sometimes missing from their work. These twin problems are manifested in recommendations that lack both sensitivity and depth. This could be due to lack of experience, lack of induction training by an editor in chief, lack of time/energy or some other reasons. Just as reviewers may seek to leverage what power they have, so SEs/AEs may also think that their position confers some kind of positional authority that they can brandish, sabre-like, as they please. Alas, the fact that I can write about it means that it must be true to some degree, but it is certainly not behaviour that I condone and SEs or AEs who so engage will find that their positions are prematurely truncated.

Where does all this lead? Well, if you are reviewing a paper, please be kind, gentle, humble. Please help the author to do the best job they can, but please also remember that you are not the author and it is not your paper! Imagine how you would feel if you received this review. Please moderate your language: yes, you may favour direct and robust speech (and you can take it) but the author who will read your comments may have thinner skin and be more easily hurt.

In contrast, if you are an editor (AE, SE, EinC), please exercise your prerogative to be an editor properly! That means, please read the paper, form your own opinion of the paper (informed by reviewers of course) and communicate both your decision and the rationale for it, together with the recommendations of the other members of the review team, so that the author benefits from your experience and judgement. Even if a paper is rejected, I suppose that normally the editor does not want to needlessly annoy the authors. The more you can convey constructive remarks, the better.

Editing a journal is a labour of love. As I write at length elsewhere (Davison, 2024), there is a lot of work involved, not just with the papers that are submitted, but also with their authors and the various members of the review team, all of whom have personalities that need to be tended carefully. Alas, not everyone subscribes to the highest of ethical standards, and some display behaviour that may make you wince in disbelief. If you are faint of heart, editing may not be for you.