Why peer review should be fun but can make you cry

The life of an academic is very much focused on and surrounded by the process of peer review.

With peer review, I mean making and receiving comments on manuscripts that we have written or have been asked to read and assess for scientific journals.

Most of us review papers for scientific journals in the hope that we can support a rigorous scientific process in producing knowledge that has been considered from all angles.

I am yet to go through a peer review process that did not add value and improve my own writing and communication of my research.

But it is a tricky process in many ways and just having gone through it again, I wanted to reflect here on why we do peer review and how each of us could perhaps structure feedback in ways that is constructive and has the potential to increase the quality of work that is being assessed.

The challenges of conducting peer review

Peer review has a tremendously important role in keeping science honest and to make sure that what is being published is an adequate account of an issue under scrutiny.

But sometimes peer review and in particular getting review comments back is far from fun. Most academics winch when they open the dreaded email and know that time has come to read what someone thinks about our research.

The opinions can indeed be widely divergent. For example, in a recent peer review of a journal article I am writing with colleagues, we received 11 pages of comments from 3 different reviewers.

Many of these comments were helpful as these sought to clarify our terminology, analysis and presentation of the results.

However, as we have to respond to each comment, the 11-page document turned into 25 pages, and this is not including the changes we made in the revised manuscript.

This time around I felt that I was writing two new papers: revising the actual manuscript and then writing responses to the reviewer comments. 

One author noted that our conclusions were the best piece of the manuscript whereas the other argued that the conclusions were merely sweeping generic statements that lacked the insights presented in the paper.

One reviewer was very happy with the terminology whereas two others wanted us to use similar terms they would use.

In the peer review process, we have a double-blind peer review: neither the author/s or the reviewer know each other’s identity, or blind peer review where the author/s name is shown but reviewer names stay hidden.

Both of these have been put in place in order to maintain some level of integrity and reduce bias.

Although at times one can easily guess who the author and the reviewer are: just count the most number of publications starting with the same last name.

But anonymity can also lead to overly negative comments and sometimes to even outright manoeuvring especially when people work in the same area of research and might be competing for the same grants.

My first cry and peer review

When I was submitting my very first journal paper, I had worked hard in condensing my master thesis research into a journal article that was submitted as part of a special issue.

By the time I got my first review comments back, I was holding back tears when a professor at the department read them out to me.

Apparently my work added no new value to the field and was not publishable.

All the other submissions in the special issue moved into revision (opportunity to revise before final submission and publication).

Except mine.

I was devastated. At the time I did not even know how to submit a paper to another journal (I was fresh out of master’s degree and didn’t even know what the process looked like).

So I just let it go.

My next attempt was a journal paper based on a project I had been leading in Africa.

The peer review comments were conflicting: one reviewer thought the paper was very good, another thought that literature review was excellent but the results needed more structure, but the third…

The third reviewer said my paper was basically rubbish and there was nothing new or valuable in it.

The editor often intervenes in such cases and points out which comments the author needs to address. Here this was not the case.

But with the support of my professor, I persisted with the comments and re-wrote what needed rewriting and learned the submission process by heart.

And alas, my first paper was shortly published after I started my PhD.

I was so full of joy and pride that I had managed to publish my first paper.

But part of the experience also left me upset again. I could not understand how my paper did manage to see the light of day given the very conflicting comments.

With more experience, I do now understand the process better however and I actually find reviewing papers exciting and interesting, and quite fun.

Why reviewing is rewarding

There are several reasons why reviewing is actually a good thing to do (and we do it for free btw).

Good thing about reviewing other scientists’ work is that you get to see what is novel and new in the field and the kinds of submissions that are in the field at that point in time.

This helps in distinguishing your own thoughts often about which directions are emerging in a field of research, and keeps you on your toes as well as to what a robust scientific paper should look like.

I find that reviewing papers is actually a good way to keep up to date with the literature and also some of the more innovative methods that authors have been testing in the field.

I encourage my students to review when they can as this gives also access to how to situate their own research in the field and what they could be thinking about.

Also engaging in a robust and helpful review of others work helps you to become established as an expert and people might seek your advice also in the future when it comes to seeking particular experts for committees or initiatives.

At the same time, when I review someone’s work, I also have the responsibility of making sure that whatever is published contributes to what has already been discovered before.

This of course is a grey area in many ways because novelty can be claimed in many ways: for example a new method, a new angle, a new geographic region or sector.

But the main aim is to increase the quality of science.

How to be a helpful reviewer

Although this post has focused on the scientific peer review process, many of these relate to the basic principles of giving feedback on reports, which are essential skills in leadership and management as well.

In reviewing, I always look at several things that can improve the manuscript:

  1. Clarity of writing/ideas: is the writer explaining his or her logic clearly, are there concepts and sentence structures that make it easy to understand what the main aim of the manuscript is? Do the methods, results, and discussion contribute to the main aim of the article?
  2. Novelty of research/ideas: how is the manuscript contributing to knowledge in this area? Are they citing the main sources and building upon what is already known? Are the methods novel and insights different from what has been published before?
  3. Structure of the manuscript/report: are there specific elements missing from the structure? Is the flow between sections good?
  4. Recommendations: what kinds of recommendations emerge based on the research? How could these be implemented innovatively? What are the next steps/knowledge gaps identified?

On top of these, the tips I would give for someone starting out in academia is to understand the field (who has published what, which are the major debates you wish to contribute to) as this helps you to situate your own research in a niche or an area that has been overlooked.

This is particularly why PhDs are often required to spend most of their first year just going through literature and finding the gaps. In some areas this is easier and in others it is not.

Mind you, although I know all of this, it does not mean that all of this comes naturally to me. I find it even more difficult to judge my own writing but I do try to keep these guidelines when I start drafting articles and reports.

The point is that to the contrary of science being portrayed as a mere rational non-emotional process, peer review in particular raises often lots of emotions (not necessarily always tears, mind you).

And that’s ok because it also signals how much our work and that of others means for us and that we are putting our hearts and souls in the papers we write and science that we produce.

 

Leave a Reply