« Rotations | Main | Thanks, and Happy March! »
Sunday, March 01, 2015
Why Do Peer Review?
A recent post by Steve Bainbridge raises a nice issue: how should we think about peer review? Traditional peer-edited legal journals have established procedures (JELS pays honoraria and blinds; JLS pays but doesn’t; JLEO has fantastic peer comments, etc). But in the last five years, most of the top student-edited journals have moved to some kind of peer system – and many of us are now routinely asked, after a student-led process, to review for publication. That peer review is never paid, and very often professors are asked to review for journals that have never accepted them. *cough. Yale Law Journal I love and hate you. cough* That can frustrate even non-curmudgeons. Why do it?
- For institutional credit. I’m aware of no school that gives formal credit for these student-edited peer reviews. Are you? If so, what does it look like?
- For Law Review credit. One explanation I’ve heard for doing a review for, say, Harvard Law Review, is to motivate them to feel that they owe you at least a rejection on your own work, instead of a magnificent silence. In my experience, there’s some truth in this: doing peer review gives you the email of an AE, and credit with that person. I routinely have succeeded at being at least read by a journal I’d just done peer review with. I haven't yet moved from a read to an acceptance. But I did get a personalized email from HLR once. It mentioned that they had an unusual number of great articles that cycle, which meant that they couldn't publish even good work like mine. I thought that was nifty! Of course, the credit isn't merely transactional: being a peer reviewer means you are an “expert” in the field, which should provide your article some kind of halo effect. Of course, this feeling is a quickly depreciating asset, and never rolls over from year-to-year. Use it or lose it!
- For the love of the game: For those of us who think that student journals should move exclusively to double-blind review, with faculty participation is a veto, participating is a price we should gladly pay. The problem is that the system isn’t perfectly constructed. Law journals should insist that peer comments will be conveyed to authors – this makes the comments much less likely to be petty (“cite me!”) and more likely to be constructive.
Bainbridge argues against mixed peer review systems, but none of his objections strike me as particularly relevant if the process is "student-screen, peer-veto." That is how I understand the system to work at SLR, YLJ and HLR. I don’t know about Chicago – I would’ve thought their selection involves a maximizing formula and ended with a number.
Posted by Dave Hoffman on March 1, 2015 at 12:36 PM in Peer-Reviewed Journals | Permalink
Comments
Peer review is not all that its cracked up to be vis a vis screening abuses. Personal example - I had an article rejected within hours by the editor of this UK based elite peer reviewed journal. After being rejected literally within 2 or 3 hours I noticed on his webpage that his publications - his writings had taken the opposite position of the paper he summarily rejected. So I figured this was his bias showing as he didnt even send it out for peer review. And within 2 weeks, I had accepted an offer from a very very high end US student edited journal. So I dont believe the peer review process is so great - there is the very real potential of bias/prejudice.
Posted by: my view | Apr 14, 2015 4:39:22 AM
Personally, I think the student screen step is the problem. If students are screening, then they are going to continue looking for weak signs of scholarly ability, such as the logo on the letterhead. The existence of a faculty backstop that can veto articles that slip through the screening process is irrelevant if what we're primarily worried about isn't false positives (i.e., the mediocre piece by a renowned scholar that makes it through), but false negatives (i.e., the terrific piece by a junior scholar or practitioner that never gets a second look). The non-blindness of the student-run law review system hurts the profession much more in the false negatives than the false positives, IMHO.
Posted by: Jack | Mar 2, 2015 10:50:33 AM
That's a good point. I should add that I never give a thumbs up or thumbs down and say that the students will have to judge based on what else they have.
I do always try to make clear which critiques I think the authors can correct and which they are likely stuck with. That said, as the Bainbridge post notes, because the publish decision isn't conditioned on the change, perhaps no good comes of such suggestions for improvement.
Posted by: AnonPeer | Mar 1, 2015 2:06:45 PM
I would feel better about the law review "peer review" system if it built in an important feature of peer review in humanities journals: a "revise and resubmit" option. Law reviews ask for a hard yes or no. But many times a fair assessment lands in between, at least with a manuscript that has a methodological flaw or is missing something important, and you think the rest is good and important enough, and the author seems talented enough, that you think they would likely be able to revise it to address the problems. This can lead to a negative assessment when you'd rather assess the article again after revisions. The speed of the law review process might make this difficult, but why not at least take a week to let authors respond to criticism - either by revising or by explaining what revisions will be made or why the criticism is wrong. That more engaged form of peer review might actually accomplish peer review's critically important function of improving scholarship.
Posted by: Mary Dudziak | Mar 1, 2015 1:54:49 PM
I think the answer has to be for the love of the game. I continue to do it (and I've reviewed for Yale, Chicago, and Stanford - apparently I'm not good enough to make it onto Harvard's radar) because I think there are great articles that get passed over and weak articles that get picked. So I try to honestly point out the strengths and weaknesses of articles (and especially point out novelty and overclaiming) so that the students can make a more reasoned choice.
I've also been on the other side of it, and have lost out due to peer review that was lukewarm (in part due to my own overclaiming). Somehow, I prefer that world to uninformed student preference.
Posted by: AnonPeer | Mar 1, 2015 1:45:21 PM
The comments to this entry are closed.