There are two documents here, the first written for the world at large and the second written in response to a specific question and thus written in the first person. They overlap in content somewhat, but not so much that both may be useful to someone willing to read through the entire page.
A reviewer should have at least four goals in mind in reading a manuscript and preparing a review for the editor(s) of a journal:
1) Providing the editor(s) and author(s) with constructive suggestions to improve the manuscript. That's hopefully what you can write, but sometimes the list has to include incurable flaws too. It's a good idea to enumerate these comments and suggestions, so as to facilitate future communication between author(s) and editor(s) (e.g., so that the authors can write in a cover letter for their revised manuscript that "We have addressed Point 3 by clarifying that . . . .").
2) Providing the editor(s) with an evaluative opinion as to whether the manuscript should be published as it stands (a rare finding), published with minor revision (a common finding), published with major revision (a common finding), or rejected (not a common finding). Specific justification should support a recommendation of rejection.
3) Writing a review that the author(s) will find constructive, rather than insulting. It's tactful, for example, to critique the manuscript, rather than the authors or their work. It's also tactful, and often true, to say things like "I may be missing something important here, but I don't understand . . . (some specific item). At the very least, clarification is needed so that other readers don't run into the same problem".
4) Providing some quality control. For example, I often look at tables of data to see if figures presenting the data agree with the tables. Astounding things can emerge.
On the other hand, there are tasks not expected of reviewers:
1) Correcting typographical errors and editing for style.
2) Checking correspondence of references in text with the list of references cited.
3) Checking adherence of the manuscript's format to the journal's format.
Here's a boilerplate review:
Dear Dr. __Editor's name__,
This is my review of Manuscript #_(Editor's reference number)_ by _(authors)_ on _(at least a hint of the title)_. The manuscript _(summary of the manuscripts key contents or conclusions)__. I recommend (one of the options in Item 2 above), for the reasons discussed below.
In summary, ___________. The comments listed above are meant to be constructive, and I hope they are of use to the authors as they revise their manuscript.
Sincerely . . . .
You will also usually be asked to fill out a review form, on which standardized responses will be requested.
I look for an introduction that says what the paper's topic is, and why the paper's topic is relevant.
I look for a section of data, or results, or something concrete, reported as evidence.
I look for a conclusion or inference made from those results. I then see if those conclusions and inferences seem reasonable, given the data.
As I read, I make a note of anything that seems unjustified, or contradictory to my understanding of the world.
I see if the title reflects the content and point of the paper.
I look at the figures to see if they are relevant, efficiently drafted or clearly photographed, and clearly labelled. There are almost always some that don't meet at least one of these criteria, most commonly the third one.
I do spot-checks to see if tables, figures, and text are consistent. For example, I look at a table of data to see if I can match two or three analyses listed there with two or three symbols on the figures. If I check a few and all seem OK, I quit. If not, I start checking more tables and more figures to see how many of them have inconsistencies.
I don't feel obligated to check the references against the text. I similarly don't check the format of the references, or the format of anything, for that matter. That an editor's (or copy editor's) job. I don't feel much obligation to worry about grammar and syntax, unless I'm feeling particularly editor-ish (which, as you know, I do sometimes).
I look at the references to see if there are any references from the last ten years, or any references older than five or ten years. Failure to meet the first condition suggests a problem that ought to be reported to the editor, and the failure to meet the second probably does too.
I then write an itemized summation of my findings. I say "itemized" because making specific statements, rather than general subjective comments, will carry much more weight with the editor, and because itemizing with numbers makes it easier for the editor and author to correspond later - the editor can tell the author to be sure to address Items 1 and 3 in your review, and the author can send back a revised manuscript saying that they have responded to your Item 1 by deleting something, to your Item 2 by adding something else, etc.
I try to write a review that is not insulting but firmly states its points. I never say the authors are stupid or misguided; in fact, I try not to mention the authors at all. I may say that the conclusions don't follow from the data, that the figures aren't clear, or that the data have flaws (e.g., percentages don't add up 100), but I never (I hope) say that the authors have made ridiculous conslusions, that the authors have prepared their figures in a sloppy way, or that the authors have mishandled their data. I try to write about the manuscript, not about the authors.
I try to write a review that admits that I could be wrong, and that the authors may have their reasons for things. I try to say things like "I may be missing something here, but I don't think X justifies Y", rather than saying "Only an idiot would say X justifies Y". I try to say things like "It seems to me that Figure 3 could be more effective if it included . . .", rather than saying "Figure 3 is complete mess and needs . . .". I try to say things like "The authors might want to look at papers by so-and-so and . . .", rather than saying "I can't believe the authors are so ignorant that they haven't cited . . .". I may think the second thing in each case, or at least wonder, but I try to employ the first, rather than second, usage.
I begin the review with "This is my review [manuscript title] by [at least the first author et al.] for [journal name], and I work in the manuscript number in if the journal uses such things.
I conclude the review with a summary of my feelings about the manuscript, a statement of what I think should happen next (immediate acceptance, acceptance after minor revision, major revision and re-review, rejection with a suggestion as to a more appropriate journal after revision, or rejection). The first and last are the least likely. I then say that I hope my comments help the authors in their revisions, and that I wish them well.
The range of quality of manuscripts submitted is amazing. Some are ready for publication without modification. On the other hand, I once reviewed a manuscript that began with the heading "Introduction" and never had another heading - seemingly the author was so unfocused and disorganized that he or she never realized that they had just meandered on through the pages without any organization. I have been given at least one manuscript that wildly violated each of the expectations implied in my list of "to-dos" above. Someday you will get a manuscript about which you'll say "Can this really be as bad as it looks", and if it does so after three or so examinations, you have to tell the editor what you've found. You'll have done the world a service in saving other people from finding the paper in a journal and having to reach the same remarkable conclusion each time.
Email to Railsback (email@example.com)
Railsback's main web page
UGA Geology Department web page