The community site for and by
developmental and stem cell biologists

Peer review – why we need it and what we need

Posted by , on 16 December 2024

Hopefully some of you will have seen the recent editorial in Development on our approach to peer review. If you haven’t read it yet, please do take a look. In it, James Briscoe (the journal’s Editor-in-Chief) and I discuss some of the initiatives that the journal has taken to try and support authors through the peer review process – including, most recently, encouraging authors to include a ‘Limitations’ section in the discussion of their article, giving you an opportunity to lay out explicitly the scope and extent of your study and, where appropriate, to respond to referee concerns by acknowledging them rather than addressing them experimentally.

Off the back of this editorial, James has also written a blog post that I’d really encourage you to read. Entitled ‘In Praise of Peer Review‘, James sets out why he believes that peer review (in some form) is an invaluable and irreplaceable part of scholarly communication. Alongside the debate that’s been going on around eLife’s exclusion from Web of Science (and subsequent decision to send a partial feed of articles for indexing, the piece has generated some discussion on social media both around whether peer review actually works to guard against publication of fraudulent, sloppy or otherwise dubious papers, and around the degree to which it actually helps to improve papers. I think James has done a great job of setting out the ‘why’ of peer review, but here I thought I’d give my view on the ‘what’: what should a peer review report comprise?

But before I start, let’s remember that – in the majority of cases at least – peer reviewers are both 1) highly knowledgeable in the field of the paper they’ve agreed to review and 2) well-meaning. Yes we all know of cases where papers have been sent to referees that weren’t sufficiently expert or who set out to block publication for political or petty reasons. But these are in the minority – most reviewers are competent to do the job they’ve been asked to, and they want to do it well. And they do it for little or no reward, because they believe that it’s an important part of their responsibility as a member of the academic community. If or how they should be rewarded is a whole other topic that I won’t get into now, but I am incredibly grateful for their dedication.

So, what do I want a referee to do?

  • Firstly, I want a referee to be respectful. Remember that there are people behind the data and – before hitting the ‘submit’ button on their report – pause to consider the potential impact of your words on the authors, particularly the students and postdocs who’ve actually done the work. At Development, we’re very fortunate that the vast majority of referees do abide by this guidance, but that’s not to say that I’ve not come across the odd report that felt overly combative or dismissive in tone – and that’s not OK.
  • Secondly, I want the report to be reasonable in terms of the amount of additional work requested. Think about the amount of time (and money!) that might be involved in addressing any particular point and ask how important that point really is to the main story of the paper. Which leads me on to:
  • Thirdly, I’d ask the referee to focus primarily on addressing the question ‘do the data support the conclusions?’ and not ‘what could the authors do to make the conclusions more interesting?’. While it’s very useful to get expert opinion on how important/relevant/useful/important the paper will be for the community, it’s primarily the editor’s job to decide on whether the paper is – in principle – appropriate for the journal in question.
  • And finally, I want the referee to be honest about what aspects of the paper they can and can’t (or even did and didn’t) assess. Are you able to judge if the authors have used appropriate statistical analyses? (And if so, did you actually check?!) If the paper contains computational work, do you have the expertise to assess it fully? If the authors deposited data, did you look at it? We fully appreciate that referees can’t always be experts in every area of a paper – particularly an interdisciplinary one – and we try to recruit referees with complementary expertise, but it’s really useful to know what you did and didn’t review.

Most reports I read (and I read a lot!) do largely follow these guidelines, but there is still a definite tendency for a referee report to read a bit like a shopping list of potential experiments and textual revisions. Experienced authors can often read the nuance to decide which points to tackle experimentally, and good editors will (either pro-actively or in response to author queries) help to navigate the revision process. But referees can also do their bit to shepherd papers through the often all-too-painful process of publishing by remembering that there’s both a financial and a temporal limit to how much a group of authors can (and should) do to revise a paper, that a single paper can’t solve a whole research question, and that their opinion isn’t necessarily any more valid than that of the authors (or, for that matter, the other referees).

We could discuss ad nauseam the benefits and problems of pre-publication peer review in its current form (and I frequently do!), and alternative models are beginning to emerge that can act in parallel to, or even replace, our current system. But let’s also think about the little steps that we can take to make the current system less onerous and more constructive – thus easing the path to publication.

Thumbs up (3 votes)
Loading...

Tags: ,
Categories: Discussion

Leave a Reply

Your email address will not be published. Required fields are marked *

Get involved

Create an account or log in to post your story on the Node.

Sign up for emails

Subscribe to our mailing lists.

Most-read posts in November

Do you have any news to share?

Our ‘Developing news’ posts celebrate the various achievements of the people in the developmental and stem cell biology community. Let us know if you would like to share some news.