« A Hair Post | Main | More on Bruderschaft »

Thursday, July 13, 2006

Peer Review in Legal Academia

In the course of some interesting and worthwhile thoughts about economics academia, Tyler Cowen offers this assessment of law professors:

The pay is higher and upward mobility is easier to accomplish. But overall the quality of intellectual debate is higher in the economics profession. In economics the return to rhetoric is lower and you can't get away with blather. Your articles are refereed by your peers, and not by graduate students (well, it depends what journal you submit to...). The academic world of law has weaker quality filters, which of course also makes it more open to new ideas.

I wonder if all of this is quite right. One regularly hears the claim that there is peer-review in the social sciences but not in law. But I think this misconceives the issue. Of course there is peer review in legal academia. It is simply post-publication peer review rather than pre-publication. Sure, a few overambitious 2Ls decide whether a professor's work goes into the West Boise Journal of Law and Footnotes or the Iowa Law Review, but tenure committees and hiring committees are largely filled with other professors, who presumably know how to recognize bad scholarship.
[Yes, I know that some professors shirk and simply look at an article's placement rather than its quality.] Many people, including my classmates, complain that the lack of peer review means that legal scholarship is worse than other disciplines. Yes, but it is also better. A free mind is prone to err, so the legal publishing model is a broad bazaar of ideas. Nearly every article can get published someplace, but citations in judicial opinions and other legal writing then form a new quality filter.
This is what Professor Cowen is getting at when he says that legal academia is more open to new ideas, and I agree. I just want to quibble. This is not because legal academia lacks quality filters or peer review. It is because those filters are imposed post-publication rather than pre-publication.

Posted by Will Baude on July 13, 2006 at 12:02 AM in Life of Law Schools | Permalink

TrackBack

TrackBack URL for this entry:
https://www.typepad.com/services/trackback/6a00d8341c6a7953ef00d834304bbc53ef

Listed below are links to weblogs that reference Peer Review in Legal Academia:

» They're both wrong from ProfessorBainbridge.com
Tyler Cowen claims the quality of intellectual debate is higher in the economics profession than in the legal academy, which he attributes to the dominance of peer reviewed journals in economics but not in law. In response, Will Baude opines [Read More]

Tracked on Jul 13, 2006 1:26:50 AM

» They're both wrong from Professor Bainbridge's Journal
Tyler Cowen claims the quality of intellectual debate is higher in the economics profession than in the legal academy, which he attributes to the dominance of peer reviewed journals in economics but not in law. In response, Will Baude opines [Read More]

Tracked on Dec 17, 2006 5:21:39 PM

Comments

The very idea that one can compare "the quality of intellectual debate" in the two fields is itself laughable. Simply because it welcomes rigorous thought about incommensurability (ala Raz and Radin) the legal academy ought to get more credit than Cowen gives it.

It's too bad that he makes such a point, because he seems quite capable of recognizing incommensurability in his recent work on Commercial Culture.

Posted by: Frank | Jul 13, 2006 4:12:52 PM

There might be a form of pre-publication review based on responses to papers posted on SSRN, or presented at workshops, but that review isn't linked (at least not in any formal or systematic way) to law review publication -- which still performs a signaling function that generates significant benefits and disadvantages for authors.

Posted by: Micah | Jul 13, 2006 1:47:20 PM

But with the advent of SSRN, isn't there now a form of pre-publication peer review? Certainly I've gotten comments from profs who have read my posted draft on SSRN and have written to tell me why they disagree with some of my points. And most profs circulate their drafts before publishing, either to peers or through faculty workshops (or both).

Posted by: Laura Appleman | Jul 13, 2006 1:26:08 PM

Ironically, Professor Cowen never submits his restaurant guide to peer review, and it is quite good. In fact, better than any restaurant guide that one needs to pay for or is vetted by editors. He has never steered me wrong.

By the same token, most lawyers can tell a credible argument when they see them, and so, the best law review articles are probably “clever” ones that help practitioners because nobody ever thought of something that way, or well-researched ones that combined a lot of overlooked sources. See, unlike most economics articles, and like the dining guide, there is a demand for law review articles outside academe, which will test and either “borrow” (with or without citation) or reject the idea pretty darn quickly.

Posted by: S.cotus | Jul 13, 2006 10:43:44 AM

And, just to jump on your point, Micah, people actually routinely get rejected for tenure in the social sciences. Not all of it based on the quality of the publishing record, mind you, but still . . .

Posted by: Ethan Leib | Jul 13, 2006 1:52:42 AM

Of course there is peer review in legal academia. It is simply post-publication peer review rather than pre-publication.

More quibbling: post-publication peer review doesn't distinguish the legal academy from the social sciences, which also review a tenure candidate's publication record. So the social sciences have pre-and-post publication peer review; whereas, in the law, most review is post-publication. The question is whether the lack of pre-publication review harms good quality work and benefits bad quality work. The competition to publish in top student edited journals, and the resulting expedite game, is pretty strong evidence that some journals are conferring significant positional goods on authors. If law students don't do a good job of conferring those goods on quality work (perhaps because of reliance on heuristics including, e.g., prior record of publishing, institutional affiliation, name recognition, etc.), then people producing good work suffer a relative loss. Even if post-publication review screens out bad work, good work is placed at a disadvantage based on factors that would be excluded in blind review.

Posted by: Micah | Jul 13, 2006 1:43:08 AM

Will writes:

"Sure, a few overambitious 2Ls decide whether a professor's work goes into the West Boise Journal of Law and Footnotes or the Iowa Law Review, but tenure committees and hiring committees are largely filled with other professors, who presumably know how to recognize bad scholarship."

Well, not really. The truth is, committees are lazy, and a lot of them don't bother to read. As a result, a serious bullshitter who has no ideas whatsoever can go very far in legal academia. Indeed, it's remakable how many very prestigious academics at top schools get ahead by writing complete crud -- crud that everyone in the field knows is crud, but that no one will ever come out and say is crud. If you don't know the field, you'll assume that Top Prof X at Top School Y must be great. But if you actually know the field, you realize that Top Prof X writes complete crud.

Posted by: lawprof | Jul 13, 2006 12:58:51 AM

May I quibble with you too? I think the idea of "post-publication" review has some promise, but you are really overstating its capacity to do its job. I guess it isn't an especially nice thing to say -- but you are really overestimating legal academics. Part of the problem is that there is no real field of specialty called "the law" -- so evaluation is very difficult to centralize. As the legal academy continues to develop and support hyper-specialities that require technical skill (L&E, ELS), I think we'll see more and more peer journals. Still, even the idea that just anyone can assess a relatively "simple" doctrinal article on the fourth amendment, I think, is misguided.

Moreover, as you well know, tenure is relatively easy to get in the legal academy, so I'd very much doubt there is substantial quality control within tenure committees. And hiring, I believe, happens like so much else in the legal academy: hype.

I want to believe in standards. But I doubt the citation index you suggest will give us much more information about quality than placement itself, since the two are so inextricably interrelated.

Posted by: Ethan Leib | Jul 13, 2006 12:43:11 AM

Post a comment