How to write a referee report?
Grad student here. I was chosen to referee a paper for a top field journal (sorry. I presented at some conference so that's how I am guessing I got onto the list.
Is there a standard guide as to how to do this. Length, what to cover, how to criticize, etc. I asked my advisor for help and she basically told me that I'm on my own.
Well, I reviewed 13 and rejected 11. There is selection: when you are an AP editors tend to give you clear decisions.
I always do my job seriously, take a lot of time to read the paper carefully and write very detailed comments. For a first submission, I've never written less than four pages. And I always let some time pass between the first draft of
my report and the final version: I want the idea and the approach taken to sink in, to be as impartial as possible, and for me it takes some time.
But I think you have a point: as AP sometimes we lack the experience to see through the contribution and understand if it matches the level of the journal.
For example, I had a rejection of a minor paper I wrote a few months ago. One referee was clearly a senior, who pointed out in a beautiful way the flaws, and suggested a minor revision. The other was clearly a junior, who got mad for a relatively secondary point and thrashed the paper with few quite rude sentences.
I know one AP, he told me that he has reviewed 17 papers so far. He recommended rejection for all of them!