« #I 🔫U: Considering the Context of Online Threats | Main | Rotations »

Thursday, April 27, 2017

reforming law reviews (a non-radical solution)

In my last post, I outlined what I saw as the incumbent players who would be likely to resist significant and comprehensive reform. To summarize, most law reviews want to self-govern; administrators view their law reviews as co-curricular activities and worry more about other important issues that their schools face; faculty complain but benefit from the current system because it allows them to avoid administrative responsibility for scholarly publications in their field; and AALS has not to date demonstrated much interest in reforming the system (though that might change, according to Brian Galle in a comment). Add to that list an additional incumbent that a comment smartly noted: Scholastica (and to a lesser extent Expresso), which profits from the multiple submission, student-run model of law reviews -- a model that peer review would most likely harm if not destroy.

This is why I am bearish on significant reform, absent an external shock to the legal academic system. And it's why I am especially bearish on the potential for the kind of radical reform that many if not most comments (especially the anonymous, snarky ones!) want: the single-submission, double-blind peer-review model that pervades most of the rest of the academy.

My main goal in this series has been to move the blame for this system off of the students -- which I often feel is where it lands, especially during submission season -- and place it not solely on faculty (where the blame also lands, especially among snarky, anonymous comments) but on a set of institutions (including faculty) that have grown up around a longstanding and imperfect system. Faculty are a part of the problem, but to view them as the sole problem is to misunderstand the system.

That said, I do have one relatively small idea that places some small responsibility on all the major players.

AALS should sponsor and host an annual or semi-annual summer conference for incoming law review editors, attended by two representatives from each law review (probably EIC and executive articles editor, but theoretically it could be representatives from the executive board). The two main purposes would be to advance the substantive components of the law review and help facilitate law reviews' coordination and cooperation. The substantive components of law review editing -- most importantly, the submission process and text editing -- are too important to leave to law reviews to develop themselves and then pass along within each law review from year to year. It's incumbent on faculty (and administrators) to try to lead new editors to view best practices and to converse with each other about ways to improve the publication of legal scholarship, and it's incumbent on administrators and the legal academy's self-proclaimed learned society to enable them to do so.

Funding such an event would be a problem. The law schools would have to pay for the editors to attend; AALS would have to sponsor and pass the costs of running the conference along to the law schools as well. This is not a good time, obviously, for law schools to have to bear new expenses. But really, given how much of the law schools' budgets goes to scholarship -- a separate conversation from this one -- spending $5000 or so per law school is a drop in the bucket for something that can at least incrementally improve the operation of law reviews. (By way of context, $5000 is significantly less than some schools spend on the summer research grant for one faculty member.) So, yes, it's a new cost, but it's an investment in improving a system that bears enormous importance to the quality and distribution of legal scholarship.

Participation would be another problem. A large majority of law reviews would have to be represented -- it couldn't just be top law reviews, or mid-tier law reviews and below. Although each law review operates independently, the main law reviews all interact with each other via the submission system and the effort by all authors to maximize the placement of their article. Not all authors and not all articles aspire to be published by the Harvard Law Review or Yale Law Journal, but many if not most do, and so the top law reviews clearly matter; but many of the articles that end up in the top law reviews are also submitted to, and often get offers from, lower-ranked journals. If faculty and administrators want to nudge reviews to adopt certain practices and behaviors (e.g., some form of anonymity in review and perhaps some degree of peer review from faculty at the law review's institution), then it's necessary to push the reviews to talk to each other and to think in a group about what would work and how best to implement any reforms. If the top reviews either skip the conference in droves or unduly dominate it, then it will have less impact and is less likely to last.

For this to work, then, AALS and the law school deans would have to be behind it and encourage -- and perhaps press, if necessary -- the reviews to attend and participate.

I could imagine several reforms that could follow, including limitations on multiple submissions and expediting (by collectively partnering with Scholasticafn1); widely agreed-upon and enforced limits on article length, especially by signalling that extensive literature and case law reviews are no longer necessary; and inter-journal communication about authors who act badly in the submission process. None of these would be a radical reform. But each would make the law reviews more professional and better for the students whose free labor make them run. It can also make the reviews appear more legitimate to scholars inside and outside the field. And, ultimately, it could have a positive impact on the substance of legal scholarship.

[fn1]: It is telling that unaffiliated entrepreneurs, rather than AALS or the reviews themselves, created a platform for electronic submission; telling, too, that the very top journals have enough resources and wherewithal to have created their own. This lack of coordination and foresight makes reform to the submission process more difficult.

Posted by Mark Fenster on April 27, 2017 at 09:46 PM | Permalink



I was asking because I think we ought to be quite critical of any solution which relies primarily on increasing the burden on students, especially when those students are not in a good position to reject the increased workload (in the instance that a school makes attendance mandatory for their EICs).


While the conference could end up making their 3L editorial duties a little bit easier, that doesn't seem to be very much of a benefit. Presumably for most students, attending the conference will require 2 days of travel. We don't have any idea how long the conference would be, but if it's a 2 day event, we're now up to costing the students 4 days of their time. Any potential benefit would have to be weighed against that cost. My experience with professional conferences has been that they're largely a net loss.

On the other hand, a conference for professors would have a lot better potential. Any benefits gained would pay dividends for years, rather than just a few months. I also suspect that a loss of 4 days for a professor is a significantly lower cost than a loss of 4 days to a 3L. It also has the nice benefit of shifting more of the burden to the people who being paid -- rather than paying -- to be there.

Posted by: Derek Tokaz | May 1, 2017 12:04:36 PM

Derek writes:

Orin, After these professors meet to formulate non-binding best practices, who are they providing recommendations to? Is it the student editors, or the professor authors?

I was thinking editors, but why not both? They could say what journals should do and what authors should do, effectively offering a code of best practices, norms and ethics for both that could help influence conduct.

Posted by: Orin Kerr | Apr 30, 2017 2:46:11 AM

"Regarding his first comment, I agree that intervention by important authors and faculties could have a significant influence that could influence top journals and then filter down. I wonder, though, if those authors have either the interest or the incentive to intervene. The current system is one that benefits the famous and well-positioned, or at minimum gives the appearance of benefitting them. "

It benefits them, but with a kind of benefit that they don't much need anymore. Someone looking to get a TT position or lateral to a better school would love nothing better than to be able to place whatever they like in a top journal no questions asked. But when you are already in a named chair at Harvard or Yale it's less important.

Posted by: john | Apr 29, 2017 1:23:03 PM

I have to partially disagree with something Professor Kerr said above, but it's from a different point of view and is not really in opposition to the direction of what he was saying.

"[I]f you think law reviews should be written for academics, then law reviews are terrible because law student editors are not subject matter experts and can’t adequately spot top academic work."

Perhaps if you think law reviews should be doctrinal, this is true. Student editors are not subject matter experts in, say, the historical development of the First Restatement of the Conflict of Laws, and can't judge articles on that sort of topic.

The converse implication, however, is emphatically not true, especially once one reaches outside of strictly 1L doctrinal work. If a piece is "Law and..." or in an intensely fact-bound area (e.g., law concerning such mundane things as DNA testing, which we've had a Supreme Court justice admit he understood none of), it is equally likely that law-review editors are at least equally prepared as subject-matter experts as is Joe Lawprof. Ironically, this is even more true among the "older" law students -- those with substantial non-law academic and/or professional experience prior to law school -- who are more likely to be law-review editors below the magical top-ten journals (if only due to the institutional inhospitality of the top law schools to those students). It's not just science, either; I specifically recall one article at the intersection of anthropology and law from my time as an Articles Editor that benefitted from two editors' existing familiarity with academic controversies in anthropology that would have been terra icognita to "regular" law faculty...

And marrying these into the process at specialty journals, such as a hypothetical Biglaw Journal of Law and Information Technology, just makes this more apparent.

I would thus soften Professor Kerr's remark. I would hedge. I would be more, umm, lawyerly... in the bad way the public thinks of lawyers.

Posted by: C.E. Petit | Apr 29, 2017 12:21:22 PM


After these professors meet to formulate non-binding best practices, who are they providing recommendations to? Is it the student editors, or the professor authors?

Posted by: Derek Tokaz | Apr 29, 2017 7:05:42 AM

Thanks for the reply. To be more direct, I think the National Conference is likely what comes of this idea: somewhat supported (but not at greater expense), somewhat attended (but less so by "elites"), no broad reform. A few aspirational best practices is not likely to resemble the suggestions in the original post or the comments.

I agree with 2013 Orin that consideration is due the diverse audiences of law reviews. More broadly, I think you could say more about the problem you find most serious and that you're trying to cure: we know from your posts that reforms are hard to accomplish, but it's harder yet to evaluate without knowing what problem looms largest. Could be lack of peer review, or length, or other things, but the right solution (and the right process to actually accomplish that solution) might well vary.

Finally, agree with 2017 Orin (he is consistently agreeable) that a shared statement on best practices by leading legal academics is an intriguing idea, whether in their voices as authors or readers of scholarship. I suspect it would be easier to manage on some issues (external evaluation, anonymity, or maybe even length) than others (like refraining from simultaneous submissions or submitting to home reviews). A pledge to submit only articles meeting particular criteria, or to submit only to reviews following a particular set of practices, is less likely to get a critical mass and more likely to come across as imperious or intermeddling . . . though it might be entertaining to watch for defections, as with the clerkship hiring plan.

Really, folks are already creating a more positive, subtle form of pressure by collaborating on alternative, often peer-reviewed, journals, leaving it to student journals to adapt in return . . . and some are. It's just slower and more haphazard and less universal than any imagined industry-wide change.

Posted by: Ed Swaine | Apr 29, 2017 5:21:48 AM

Admittedly, my recommendations may have been swallowed up by the volume of comments on various posts. I refrained from repeating them here because (a) the same thing said over and over can become tiresome, and (b) I thought it worthwhile to keep the conversation focused on the original proposal, at least for a bit.

To reply to Fenster's point. Peer-review is not sufficient for law reviews to be taken seriously by other disciplines. However, it is necessary. The policy's problems do not detract from its necessity. However, the legal academy is actually well-placed to fix some of peer-review's problems. Most of peer-review's problems come primarily from a lack of qualified reviewers. In the legal field, the academy has an entire army of subject-matter experts in judges and lawyers and professors. If these professions were invited to participate in peer-review, I believe many of them would accept. This would involve the profession in legal scholarship, and do a far better job of getting profs to write for the field as a whole than student editors. In addition, practitioners would have a far better chance of acceptance, thereby ensuring a more diverse literature.

However, my initial suggestions don't actually have anything to do with peer-review. My initial suggestions are:

1. Non-negotiable world count (15,000 words is ideal)
2. Complete anonymity

Both are easily instituted and do not affect law reviews institutionally. In addition, both will be necessary for the inevitable switch to peer-review. Instituting both of these solves the problem of legal scholarship being overblown, over cited, overwritten, disguised political screeds. In addition, it solves the letter-head bias that disproportionately affects legal scholarship.

Posted by: YesterdayIKilledAMammoth | Apr 29, 2017 12:52:28 AM

I appreciate Mammoth's insistence on peer review, but I assume s/he would agree that peer review is not sufficient for legitimacy -- plenty of journals and fields use peer review and have even less visibility and impact to scholars outside their fields than law reviews. I am not convinced that it's necessary, especially given peer review's many and well-reported flaws, but reasonable minds could differ on that point. But the argument neither responds to the point of my posts nor is it helpful in pushing an admittedly strange, entrenched institution in a different direction.

Regarding Ed Swaine's point, I don't think the NCLR will change law review practices; a quick glance at its member journals reveals that the "elite" reviews eschew the organization and its conferences. Which is not to say that it's bad or worthless. Two students from my school's law review attended this year and I think they found it helpful, although they attended this year because its venue was within driving distance, not because they deemed it an essential event to attend. A much larger and more representative event would, I think, lead to a more effective sharing of best practices, as you suggest.

Derek Topaz: I don't agree that editors who would attend the conference stand to gain "very little" from it. Even beyond the inevitable networking opportunities, these are students who would be beginning a year in an editorial position that will, as you say, take up a lot of their time. Having the opportunity to share ideas from across journals and create channels of communication that they can use for the year ahead is likely to be valuable.

I agree entirely with Orin's second comment, and I should have noted that "legitimacy" for law reviews extends beyond the scholarly world and into the profession. Regarding his first comment, I agree that intervention by important authors and faculties could have a significant influence that could influence top journals and then filter down. I wonder, though, if those authors have either the interest or the incentive to intervene. The current system is one that benefits the famous and well-positioned, or at minimum gives the appearance of benefitting them. I'd like to think the fact that it does so, and is so widely known to do so, would inspire institutional and intellectual leaders to try to reform the system for the benefit of their field and own work. But their failure to do so suggests that the problem requires an institutional response.

Posted by: Mark Fenster | Apr 28, 2017 10:58:11 PM

Mammoth writes:

"Law professors need to decide two questions. First, do they want to be part of the larger academic world? And second, do they want to influence the practical world? If the answer to either of these is 'yes', then systemic change is needed."

But what if law professors want to be both part of the larger academic world and also want to influence the practical world? My take from 2013:

In my view, law reviews are easy to criticize because they are a strange hybrid. They purport to serve two audiences at once — both legal academics and the bench and bar. Their hybrid status satisfies no one. If you think law reviews should be written for judges and practicing lawyers, then they are terrible because they generally are not relevant to the practice of law or the kinds of issues that judges regularly encounter. On the other hand, if you think law reviews should be written for academics, then law reviews are terrible because law student editors are not subject matter experts and can’t adequately spot top academic work. Both of these critiques are entirely fair within their assumptions. If you pick your audience, you have your critique.

But they are also in tension with each other. For example, those who criticize law reviews for their lack of real-world impact may want to think twice about the role of students in the process. As between student editors and faculty editors, student editors are probably more interested in real-world impact and accessible writing than are elite academics. As a whole, the knowledge that student editors will be selecting articles probably pressures legal academics to write scholarship that is more easily understood by lawyers and that has more real-world relevance and impact than they would if the journals were all edited by professors. This doesn’t mean that the status quo is the best alternative, of course. But it does mean that you need to start by defining your audience, and then consider what works best for the audience(s) you have in mind.


Posted by: Orin Kerr | Apr 28, 2017 9:12:03 PM

One idea would be for a bunch of big-name law professors to get together and agree on a set of best practices for law reviews. It wouldn't bind anyone, of course. But I suspect that if it were signed by a lot of bigwigs -- think Cass Sunstein, Jack Balkin, Mark Lemley, etc. -- a lot of law review editors would read it and take it pretty seriously. And it would be free, too.

Posted by: Orin Kerr | Apr 28, 2017 8:53:27 PM

Real reform is going to have to carefully (and honestly) look at the incentives behind how law reviews work.

The primary beneficiaries of the system are the professors who get to publish their works there. Second to them are the students who can use it as a signalling mechanism to employers (though this is perhaps subject to a bit of an arms race, with a negative presumption being applied to people who didn't serve on a journal).

The people most hurt by the complaints we hear, and who therefor stand to gain the most from reforms, are the professors.

With that in mind, it seems like perhaps a non-starter to look at a solution that requires more time and effort (and possible personal expense) from the students who stand to reap very little reward from the change. I don't know how you'd get the students on board other than to just have the school formally require attendance at the conference.

Posted by: Derek Tokaz | Apr 28, 2017 10:55:07 AM

There is already an annual National Conference of Law Reviews -- I don't know if that already addresses some of your agenda.

If there are larger problems to address, the solution of "let's hold an annual conference" might sound like "let's put on a show." I doubt AALS or law school deans want to expend much capital, literally or figuratively, on it. It is, as you say, expensive, and as you warn, it's unlikely in practice to secure widespread participation in a coordinated solution. Personally, I doubt that "widely agreed-upon and enforced" standards on things like article length are anything to aspire to, unless perhaps you mean something like the development and sharing of best practices.

Posted by: Ed Swaine | Apr 28, 2017 4:34:28 AM

Anything short of peer-review will not make law reviews more legitimate to outside disciplines.

While, I applaud at least one proposal to get us going, the proposal begins with funding and participation problems--which are death knells for any meaningful reform.

Law professors need to decide two questions. First, do they want to be part of the larger academic world? And second, do they want to influence the practical world? If the answer to either of these is 'yes', then systemic change is needed.

If the answer is no, then we'll continue to watch the growing obsolescence of legal research.

Posted by: YesterdayIKilledAMammoth | Apr 28, 2017 2:06:21 AM

The comments to this entry are closed.