Logo of the Comparative Ecology Group showing a tree with the trunk splitting into the face of a bird and the face of a human

How to approach reviewing for peers

What to consider when giving feedback on the work of others

Assessing the work of others is a major part of academic life. It can start with troubleshooting for people you work with, giving feedback to friends, to more official roles of being a reviewer of papers and grants.

Here, I provide some of my thoughts on what you might want to consider and links to helpful advice others have put together.

First, if you are new to the process, it can help to familiarize yourself with the publishing landscape. You might have ethical considerations to only invest your time and effort for certain publishing outlets. Yes, reviewing can help the authors of the work you are assessing and you contribute to checking the scientific literature. Depending on where the work is coming from though, you are also providing a service to publishing outlet that might profit from your work, have standards you do not agree with, or you might prioritize open science and therefore not review for closed systems.

When drafting the review, keep in mind who you write for: you want to give feedback to the authors, but consider that the first person to read your review is the editor.

For the authors, find a format to keep it professional, focusing only on the content of the manuscript and assuming that people make honest mistakes. Unprofessional peer reviews have actual consequences, they can harm and stress people, with disproportionate consequences for historically excluded groups. Make sure that you are not suggesting the authors to commit ethical quandaries such as p-hacking or h-arking. That can be difficult if you review a completed study, where you might find that the methods are not suited to address the questions the authors wanted to answer. One way forward could be to consider switching to reviewing before a study is conducted. If you want to learn more about registered reports, here is a blog I wrote with Corina Logan.

For the editor, you might want to put in a short summary to indicate that you covered the key points of the article in your review. List any parts of the manuscript you did not check in detail because you did not have the expertise. That’s totally fine: you can’t be expected to understand everything about papers, especially if they are the result of a large-scale collaboration. My guidance is that as reviewer your job is to check whether the manuscript is scientifically sound. The decision whether a manuscript should be published in a particular journal is up to the editorial board. You can however help the editor to make that decision by covering some general points. For example, if you have questions about the methods or the results, you might want to indicate whether you think the authors can address the changes. If so, would that involve rewriting, changing analyses, or do you think that with this approach the authors might not be able to answer the question they set out? Questions about impact are subjective: my advice is to focus on who you think the potential audience might be - specialist researchers working in a very narrow topic, researchers from a general field, researchers from different fields, the general public. None of this makes a paper better or more important, but it can help the editor to see fit to their journal (if they make such subjective decisions).

How much should you invest into reviewing? It’s a service, so offer as often as is feasible for you. There is a rule of thumb which says that you should review 3 times as many papers as you submit as corresponding authors. In theory, this would balance the service you receive. What about recognition for peer reviewing? I think you should list service activities like reviewing on your CV, but I do not support commercial systems to track reviewing activities. I see three main problems with individual-driven reviewer information system as currently implemented. First, it is unclear whether I as individual should contribute publicly funded time and information for third party commercial companies to make even more money. Second, it is unclear what benefits these services might provide to individuals in return. Metrics have serious issues as they ‘measure people’, and any quantitative single measure that is used to compare people in decision making processes is bound to not reflect what we actually want to assess. Third, benefit individuals get from these services might conflict with Good Science Practice: proper review is altruistic, see for example point 2.8 in the ‘Rules of conduct for good scientific practice’ of my current employer.

There is a debate whether you should sign reviews or not. There are arguments for and against revealing your identity to the authors. My only advice is to be consistent: if you decide to sign, always sign your name, if you decide to review anonymously, never reveal your identity. This consistency does not necessarily have to be overall, it could for example also be that you always sign your reviews for particular journals or reviewing outlets (e.g. those who publish their review) but not for others. Having consistency takes out any biases that arise when trying to decide suddenly to sign a particular review or leave your name off another. There could be reasons in particular cases to deviate, but it just leaves you open to risks of not giving everyone your best and fairest assessment.

Here are guides that have helped me as a reviewer:

My main site Overview blog posts