Monday, October 21, 2013
Et Tu, Adam? The Lazy Critiques of Law Reviews Continue
When it comes to media stories on legal education, bashing on law reviews is evergreen content. Why, it was just two years ago that David Segal was offering his own attack on legal scholarship, referring to law review articles as "headscratchers" and quoting Chief Justice Roberts on Kant & Bulgaria. (And this was my response then, in case you missed it!) So here comes Adam Liptak with his version of this tired old story, reciting the quotes & studies that have been trotted out before. I find these critiques to be based on a blend of ignorance, arrogance, and incoherence. Ignorance because they don't really seem to know what's going on in actual law reviews. The CJ's quote is a good example -- it's a caricature of a cliche about law reviews, rather than an actual observation about them. Arrogance because there is always outrage about these "amateurs" and "incompetents" getting to touch the golden prose of scholars. Sure, some journals and some editors are worse than others, but on the whole students know the Bluebook and are respectful yet challenging of authors. I have gotten terrific editing from law reviews, including a set of edits at a specialty journal that I just turned around this past month. Would some peer review be nice? Sure, but (1) there are peer review journals and (2) meaningful peer review comes in the literature to follow. When a huge number of professors do their research on SSRN, which offers no review of any kind (other than download counts), the need for peer review to separate wheat from chaff is overblown. Finally, incoherence -- because the critiques don't fit together. Law review articles are incredibly esoteric and out of touch? Then why are they being chosen by editors who almost all go on to be lawyers themselves? Law blogs are better than law reviews? I don't know where to begin with that one. There are a lot of different tropes and agendas meandering around in these critiques, and they just don't hang together. The critique of internally-placed articles based on Albert Yoon's research does raise real concerns. But this is a much more subtle point than the rest of Liptak's post.
I have a lot more to say about this, some of which I said in "Law Students and Legal Scholarship" over at the Journal of Law. But law reviews are a resource for which law professors should be grateful. I hope more law profs come out of the woodwork to defend these institutions of research and learning, or they just might begin to disappear.
TrackBack URL for this entry:
Listed below are links to weblogs that reference Et Tu, Adam? The Lazy Critiques of Law Reviews Continue:
Liptak's piece is an embarrassment. Even if all the criticism were coherent, etc., it completely ignores potential benefits. Student editing is quicker and cheaper than faculty editing, for example. And there are many more willing student editors than faculty ones, which leads to a proliferation of journals. That has costs, but it also allows more final-form, edited and published scholarship to be put out there (and collected in databases, etc.) And then, of course, there's the potential benefits of law reviews for the students themselves. I'm not saying all of these things are huge, but you don't criticize something in this way without at least mentioning it's potential benefits.
Posted by: anon | Oct 21, 2013 1:49:23 PM
Agreed with above. There is an added benefit (and it's a real benefit), which is reinforcing traditional signalling / prestige norms.
Some law and econ types with a Randian streak might say those types of benefits are not real benefits, but there's something important about requiring difficult, peacock-tail type work. Whether beneficial in some other instrumental sense or not, the benefit of maintaining some camaraderie, some standardized social trappings, some rites of passage, and some status-based system of ordering cannot be understated.
Posted by: AndyK | Oct 21, 2013 2:39:18 PM
The law review process is rightly student-centric. They are the ones subsidizing the whole operation. Ideally, serving on a journal should provide useful training (e.g., learning how to edit others' writing without alienating them), exposure to a wide variety of legal issues, and networking opportunities. In any event, if a large part of the problem supposedly is that law professors can't or won't write anything worth reading, then it seems pretty questionable that replacing student-edited journals with peer-reviewed ones would do much to help.
Posted by: EC2 | Oct 21, 2013 3:14:26 PM
How does the 'peer review' in the subsequent literature occur in the 43% of articles which are never cited (according to the article)?
Posted by: anon | Oct 21, 2013 3:21:21 PM
I would guess that Matt's response would be that the lack of citation is its own sort of peer review.
Posted by: anon2 | Oct 21, 2013 3:24:28 PM
Posted by: Matt Bodie | Oct 21, 2013 3:28:29 PM
Matt, just to play Adam's advocate for a moment: what spurred the re-tread of the tired old quotes about Bulgaria, evidence and Kant was the a) study showing that law profs are unhappy with the status quo in legal scholarship and b) Yoon's study showing bias in editorial selection. So I"m not sure it's journalistically outre to reprise these themes when there's evidence of dissatisfaction even by the fat cats who sup off student law reviews (us) or the plausibility of some corruption of merit-based standards in the selection process.
Posted by: Dan Markel | Oct 21, 2013 5:01:16 PM
I suppose that all of these judges and the lawyer of the bar just must be wrong. Yep, everyone is wrong except us law professors. Even other academics, whose mouths are gaping wide in astonishment when you tell them about the law review system, are wrong.
It must be tiresome to be right so often.
Posted by: justme | Oct 21, 2013 5:19:44 PM
Peer review is not always great shakes, at least outside the sciences. Often it's not really blind and the revise and resubmit process can take years, so articles dealing with current issues would be published long after the issue has come and gone. Many law reviews now do an informal kind of peer review, asking experts to review articles they have culled from the giant stack they receive every season.
The legal profession has a lot of problems, law schools have problems, of those I'd put law reviews at #125,634.
Posted by: AP | Oct 21, 2013 6:05:42 PM
And isn't there serious concern when, as Adam notes, law profs' tenure and promotion depend partly on where you publish? I've been lurking on the "Clearinghouse for Questions" thread, too, and those folks on the market are also worried about where they place, as they sense that appointments committees care about placement. In other words, people who need to make a living, who need to keep their jobs, who are experts in their fields, and who have toiled for a year or so on an article - all of these people are forced to toss their work out there to the 2L gods, most of whom have never taken a course in research methods?
Sure, I know all about outside reviewers, etc. But tell me that intros to job talks at your school don't start with, "X is already an accomplished scholar, with publications in Duke, U. Va., and Penn." And tell me that tenure and promotion reports, at least to the university, don't contain this (considered vital) information.
Posted by: Lisa McElroy | Oct 21, 2013 6:16:15 PM
It is easy to overstate the case against legal scholarship, and I am happy to agree that the Liptak article is overblown and simple-minded. But I do worry that legal scholarship is lacking in adequate quality control because of the lack of peer review. In this respect, I have my doubts about the claim that although student editors cannot engage in anything like rigorous pre-publication peer review, "meaningful peer review comes in the literature to follow." This process of post-publication review is rather a hit-or-miss thing, or so it seems to me. It depends on the willingness of other scholars to take the time to scrutinize critically and with rigor the scholarship of others. I wonder how often this happens.
William Stuntz was one of the most prolific and cited criminal procedure scholars in recent decades, but his work often contained broad empirical generalizations with questionable empirical support -- the kind of work that might be questioned by the editors of a peer-reviewed criminology journal, but which law review editors were evidently eager to publish, and legal scholars were sub sebsequently eager to cite. For years (really decades), the process of post-publication peer review described in this post seemed to smile on Professor Stuntz's claims -- not because anyone systematically examined the evidence supporting these claims, but because they offered support for others who wished to make arguments that were supported by these same claims. In other disciplines, however, empirical claims are not accepted merely because someone has already made them -- they require persuasive supporting evidence.
In their recent reviews of Professor Stuntz's final book largely recapitulating Stuntz's law review articles, Stephen Schulhofer and Donald Dripps have dismantled a great many of Stuntz's central empirical claims. This is the kind of thing that rigorous peer review would be more likely to catch. Perhaps we should demand rigorous peer review before we accept the claim that legal scholarship receives effective peer review after publication.
Posted by: Larry Rosenthal | Oct 21, 2013 9:42:31 PM
Does anybody have a cite for the Dripps piece that mentioned by Mr. Rosenthal?
Posted by: frank | Oct 22, 2013 8:36:42 AM
Posted by: Larry Rosenthal | Oct 22, 2013 8:47:17 AM
I agree that peer-reviewed systems tend to be more rigorous than the student-edited selection process. However, let's not pretend that peer reviewers in the other disciplines are rigorously checking each other's data. There are countless empirical papers published in which the underlying data sets and computations are never double-checked, either by the editorial staff or reviewers. Only once in a blue moon will another scholar go back and try to independently confirm the underlying findings.
Posted by: Anonymoose | Oct 22, 2013 9:00:59 AM
I'm was a student law review editor on the flagship journal at a decent law school. In my experience, articles were well-written and the articles editors did a pretty good job selecting content. But I have no idea if this is true at the secondary or tertiary journal at a low ranked law school.
I think the real problem with legal scholarship is that it's not worth what people are paying for it. The NYT article is right that law reviews are written by law professors for law professors. That doesn't mean that law review articles are worthless. After all, this is probably the case with a lot of liberal artsy academic writing, such as in comparative literature and so forth. The problem is that law journal articles are so much more expensive than articles in those other fields in terms of law school tuition and law school professor's salaries.
For that price tag, the academy should be producing something of value to larger society. It's just not worth putting that kind of money into the closed ecosystem of legal scholarship.
Posted by: Scott | Oct 22, 2013 10:49:21 AM
There's a side issue here, too:
Sometimes the law students editing the journal have more expertise in particular areas than do the professors submitting articles in shotgun fashion. As some specific examples:
* The law student who finished up a PhD in physics, decided she didn't like grant-writing and defense-industry jobs, and then went to law school
* The law student with an ABD-PhD in literature at a theory-heavy school who decided that eating would be nice after grad school and went to law school
* The law student with an MS in computer science who got tired of being a drone at BigTech and went to law school
* The law student with an MA in biology, concentrating on gene-environment interactions, who went to law school intending to work in patent law
Each of these students probably knows more about the SUBSTANCE of articles that law professors write about as dilletantes (respectively, statistical analysis, hermeneutics, the actual distinction between "method" and "process" as computer scientists understand it, and second-order genetic effects of industrial toxins... as actually observed in rejecting unfounded articles). This is a form of "peer review" that the law reviews should be proud of, not dismissive of.
At present, they're not.
Posted by: C.E. Petit | Oct 22, 2013 2:37:50 PM
I think Scott nailed it. Cost-benefit is the appropriate analysis.
Posted by: Bobbo McBobertsonton | Oct 22, 2013 8:11:45 PM
I have seen a lot of Internet discussion of this topic in the last few days, and most of the conversations about whether or not articles are useful tend to focus on whether or not they are cited by courts. I would submit that this isn't the only meaningful measure (although it may be the most convenient). My practice was and my teaching and scholarship are transactional in nature. The scholarship that is interesting to me is unlikely to be cited by courts, but I think it can be pretty darn useful to practicing transactional attorneys.
Posted by: Tanya Marsh | Oct 23, 2013 2:11:51 PM