« Donald Trump's First Amendment | Main | On the Execution Saga in Arkansas »

Monday, April 17, 2017

the eternal recurrence of law review complaints (or, why is law review reform so hard?)

My previous post concerned the narrow issue of law reviews' policy towards their own school's faculty, although some comments raised larger concerns. Towards the bottom, James Grimmelmann complained,

[t]he law-review debate crops up on Prawfs, CoOp, and/or TFL every year or two, and every time the arguments are familiar, repetitive, and tedious.

Indeed! (And as Orin Kerr noted, Grimmelmann's complaint, and the proposal he contributed, are themselves parts of the same cycle.) Many of the commentators' reforms, all of which have some merit, assume the possibility of institutional change, such as widespread, blind, and anonymous peer-review. But a major recession and funding challenges to the legal academy and higher education have produced no changes to law reviews despite the regular airing of complaints and proposed reforms. For reasons I discuss below the jump, I'm skeptical of such reform occurring in the short- or medium-term.

Let's line up the players that represent the entrenched interests in the current model, starting with the one I'm most interested in (see my first post this month). Please feel free to challenge or augment these descriptions in the comments!
  1. Law Reviews: Law reviews' traditions of self-governance, which current members and many alumni consider core to their mission, would lead many reviews (especially at the top, I suspect) to resist if not oppose significant intrusions into their operation. In a comment on an earlier post, Larry Rosenthal reported that the Chapman faculty has taken significant responsibility over its law review's content. It is one thing for Chapman to do it; it would be much more difficult for faculty at other schools to do so, and of course the Harvard administration and faculty couldn't legally do so given its law review's complete independence. A handful of journals may decide to begin some limited form of peer review, as some have, or impose some hard or soft word limits on submissions. But those are a far cry from the systematic effort to impose a stronger and broader form of peer review or some other structural reform that faculty would see as necessary to improving the current system. 
  2. Law school administrations: They provide institutional support, varying amounts of funding, and office space, as well as coordination and course credit, and yet seem to do little to help guide or manage them besides assign a faculty advisor to the organization as a co-curricular activity. No doubt administrations step in if there is a crisis or some perceived internal need to do so. They may also attempt to nudge the journal towards certain goals (e.g., diversity of membership and inclusion within the broader law school community). But it's difficult to imagine an administrator taking over or forcing significant change on a law review if the students resist and/ or the faculty has little interest in engaging with or participating in the journal's operations.
  3. Law school faculty: Faculty benefit from the very small roles they play in journal operations compared to the roles that would be required of them in a peer-review system -- although the peer-review system itself isn't terribly draining to the median faculty member in a discipline with peer-review journals unless she volunteers to take a turn editing a journal. My sense is that despite complaints about the submission system and editing, most faculty are sufficiently satisfied with the current system and have little interest in managing a general interest law review at their home school. They might be interested in participating in a field-specific peer-reviewed journal's operations, but besides a few, notable exceptions, that's not the current model for most legal academic journals.
  4. AALS: As far as I can tell, AALS in its role as a "learned society" that "promotes the core values of excellence in teaching and scholarship," plays no role in either overseeing or encouraging oversight of student-run law reviews. Please correct me if I'm wrong.

These are not players looking to initiate major change. The current system works just well enough to continue, and the various players are either sufficiently invested in it or have little interest in reforming it. Absent some external financial or institutional shock, the current system is likely to withstand reform efforts that are not generated by the law reviews themselves. And because, as I noted before, there are neither institutional structures nor traditions nor incentives for the reviews to coordinate their activities, internally-generated, systemic reform is not likely to occur either.

I think we can help the students improve the existing system, and I'll offer a few ideas in a later post. But the eternal recurrence of our complaints and their relative uselessness are caused by an imperfect system, developed over decades, that is highly resistant to significant change.

Posted by Mark Fenster on April 17, 2017 at 10:08 AM | Permalink

Comments

Orin,

1. I'm confused on your #1. Originally you described the incentive as specifically being, "providing a forum and distribution mechanism for its publication." Basically professors want to get their stuff out there, and journals help them do that, so the payoff for writing is bigger, thus the incentive is created. But then, "The incentive function to generate legal scholarship is to more than just the ability to make things available." I'm confused what you mean by the incentive function now.

2. We can easily get the benefits of a screening function without journals. The purpose of the screen isn't to ensure that bad articles never see the light of day; it's to signal to readers that an article has reached a minimum threshold of quality (one which changes from school to school). The blogosphere layer on top of the open publication network would serve the same function. The crim law prawfs would write a quarterly "Criminal Law Articles of Note" piece. Basically replace the student-run screening process with a professional-run funneling system.

3. Sure copy editing is a normal service in publishing. But the context of my comment is asking why we'd keep the arcane journal system when an open publication system could very easily exist. One reason is that professors get access to unpaid editing labor. That's a service we might not want to keep around.

4. What archiving purpose do journals serve that cannot be done just as well and quite easily by university libraries? In the event Lexis and SSRN go belly up, the libraries will have electronic and hard copies of the articles.

5. Most articles may get substantive engagement from the student editors, but most student editors are probably not doing substantive engagement with articles. The subset of journal staff who stay on for their third year will probably do that kind of work, but it's not what the 2L students are spending their time in the library working on. If journals were being designed with the intellectual benefit of the student editors in mind, they certainly wouldn't look like what we've got today. There'd be a lot more white papers and amicus briefs, rather than traditional academic essays.

6. I'll take your word for it.

7. Well hold on now. Your assertion is that yes definitely schools reputations are enhanced by managing to snag good articles to publish. But, a professor's reputation for getting published in prestigious journal ....ehhhh, maybe it helps him? Come on. When you submit to Harvard Law Review, it's not because you think Harvard deserves to have its reputation enhanced by virtue of publishing your fine work. It's because you want your reputation enhanced by having Harvard Law Review added to your list of publications.

Is there a serious argument that the main purpose of the law review system is something other than to provide professors with a proxy for the quality of their work?

Posted by: Derek Tokaz | Apr 19, 2017 9:15:41 AM

I wanted to add a voice of optimism because there's so much pessimistic commentary to this post. I very much enjoy law reviews. I almost never read them digitally, but almost always in print. I have a subscription to seventy-five law reviews, which my library liason directs to me. I read them at home, in the office, while walking down the street, at baseball games, you name it. I like the law review format. I love receiving them in my mailbox.

I consistently find deeply thought out ideas in law review pages that are a delight to read. They add to my understanding of the subject, help me with insights I would not have had myself, provide me with details of analysis for classroom and scholarship uses, and just generally make my intellectual life more joyous. Law reviews are helpful to me as a scholar and as a pedagogue. I speak of them to professors, politicians, and students.

I should add that in terms of editing, I very much enjoy working with student editors. And I say this as someone who also publishes in university presses, runs an academic press series, and often peer reviews book and article proposals.

I'd be happy to share my library subscriber list of law reviews. You need only email me and request it: [email protected]

Posted by: Alexander Tsesis | Apr 18, 2017 2:53:03 PM

There have been plenty of suggestions. I'm curious to see if there will be a new blog post dedicated to ranking these in order of importance, and then a next blog post to discuss how the top 2 or 3 can be practically implemented by the 2018 cycle.

Legal academia is controlled by a system so crony-isitic, unethical, and incestuous that I'm surprised Jared Kushner's not running it. Until it gets reformed, perhaps legal academics should stop talking about ethics. They are terrible leaders. No use being hypocrites, too.

Posted by: YesterdayIKilledAMammoth | Apr 18, 2017 2:00:46 PM

There is surely plenty of blame to go around for what is an absurd system -- from the student selection to, what is worse, the single submission window when uninformed students are making snap judgments to fill an entire volume. But Professors are also to blame, not just for their unwillingness to do peer review (many would but not all in March) but for the quality of legal scholarship. The length of law review articles is both unnecessary and inexcusable, and what passes for legal scholarship in many journals would never find a home outside of law (and explains one reason why legal scholarship is so uninfluential). Is that the students' fault? Maybe partly on the length, and I never understood why the effort to move to shorter articles failed so rapidly (I don't think it lasted a year), but it is my sense that many law professors believe the length of their articles is justified, when, in most cases, it is clearly not.

Posted by: MLS | Apr 18, 2017 1:31:49 PM

The deadweight loss of law review. The fundamental problems with law reviews are the anachronistic "submissions" process and insistence on "publication." If law reviews perform any "screening" process at all, it is not obvious to me that they increase rather than decrease quality. In any case, they waste an unbelievable amount of time. All of the relevant goals that Orin mentions could be achieved by having professors work with their own students to prepare their articles for print. And it would come with the advantage of efficiency & better pedagogy. Would law review students object? Probably. But only because they love following rules, even ones they suspect are stupid. Luckily, institutional memory is short. Faculty can just change the rules & tell the students, "It's always been that way." What they don't know won't hurt them.

Posted by: Brian L. Frye | Apr 18, 2017 11:51:30 AM

The main interest group that blocks this is the faculty, largely by inaction. We don't want to get deeply involved in the journal process because it would be a lot of additional work, probably poorly rewarded by our schools. We can't all undertake peer review with the journal submission process set up the way it is, because multiple-submission means we would be buried under requests for reviews. (Most) individual faculty can't defect from the system because sending an article to the peer reviewed journals that are available would mean waiting months for a decision, and if the decision is "no" starting all over again--something particularly undesirable for people still on the tenure track. And there is a collective action problem in tearing the law review system as a whole up by the roots and starting over again. As for students--maybe the HLR students really would fight against changes. Most students at other schools, however, want to have law review listed on their resume, but show little enthusiasm for the work they do on the journal itself. Can't say I blame them.

Posted by: hw | Apr 18, 2017 10:49:50 AM

There is a major entrenched interest missing from this discussion:

The Submission Gatekeepers (i.e., Scholastica and ExpressO)

Their interest is in the spray-and-pray approach that many legal scholars take to placing articles. More submissions, at ever higher rates, means more revenue for the Gatekeepers. But they could upend the current model with numerous measures that mostly work against their self-interest. The Gatekeepers could limit the number of journals reviewing a given article at any one time; they could treat submissions as offers to publish with the journals receiving the article, so that the journal would effectively be the one accepting the offer; they could throttle the expedite game; etc.

Posted by: BAG | Apr 18, 2017 10:39:30 AM

Owley:
Supervised training takes a labeled training data and outputs a set of weights to that can then be used as the input of another program to classify novel examples.

There's no programmer designed algorithm, it's all inferred from the data.

If idiosyncratic volume specific factors (e.g. we need a tax article) are indeed a large part of the accept/deny decision then a submission focused program wouldn't work very well. I suspect they aren't.

Posted by: john | Apr 18, 2017 9:57:36 AM

Erie,

Assuming what you say is true, it's absolutely ridiculous. That's a book-length manuscript, and whoever wrote it either, a) got turned down by every reputable university press or b) knew it wouldn't pass actual peer-review at a university press and decided not to waste their's or the press's time.

As a result, the author pawned it off on students and wasted your time. Hopefully you rejected it quickly. And harshly.

Posted by: YesterdayIKilledAMammoth | Apr 18, 2017 5:17:39 AM

Student at a T25 law school, received approval this semester to launch an "aspiring law journal." While intending to incorporate desired reforms mentioned on prawfsblawg, our first submission was 85K words....

Posted by: ErieSwiftByrd | Apr 18, 2017 3:45:03 AM

Or, better yet, just follow the example of the Journal of Legal Analysis. Just publish one complete volume at the end of the year. It'll still create a permanent record without any ridiculous publication delays.

Any prof that wants a bound version of his/her/non-gender conforming article can buy the single volume.

Some may argue that law reviews publish issues in order to keep costs down, but let's get real. Subscriptions have plummeted, and nobody except other law profs actually read the reviews anymore. So it's not like there's an interested lay (or even professional) readership for whom price may be a factor.

As for law profs, it's not like any of them suffer from under compensation. So, we have that problem solved.

Posted by: YesterdayIKilledAMammoth | Apr 17, 2017 7:53:24 PM

On Orin's #4: "Provide a permanent version and location of a past published article (so it can be readily found, correctly cited, etc if websites go bust, links rot, or multiple drafts are available)."

One alternative way to serve this function would be for the Library of Congress to add a digital "Law Manuscripts" collection, only accepting final manuscripts, and assigning cites (e.g., 2017 Law. Man. 1). I'm not sure what incentive the LOC would have to do something like this, but it's the one entity that immediately jumps to mind as having the capacity and durability to play this role.

The interface could look similar to the LOC's digital Federal Register collection, and if they added a download count for each manuscript and a sortation mechanism, I could see some professors deciding to forgo the law journal route (just as some professors occasionally post pieces to SSRN and never submit them to journals).

https://www.loc.gov/collections/federal-register/

Posted by: Jim Oleske | Apr 17, 2017 7:32:00 PM

Heavy sigh. The current system is absolutely ludicrous, complete ridden with conflicts of interest and ethical violations. On top of that, it actually encourages privileged students to leach off of the screening efforts of students at lower-ranked law reviews. And somehow, all of the adults (several of whom, no doubt, pontificate on ethics) throw up their hands and say, "What can we do? It's the system."

The system is indefensible. Period.

Posted by: YesterdayIKilledAMammoth | Apr 17, 2017 4:07:47 PM

John - How do you make that algorithm? What is the definition (or qualities) of an article that "should be accepted?" I know lots of people that had pieces rejected one year and then receive multiple acceptances of that same article the following year. Reasons for rejection/ acceptance may not mirror article quality (we already have a tax piece, the author submitted so late all journals are full, this one is too short)..

Posted by: Owley | Apr 17, 2017 3:49:15 PM

The technology exists at this moment to build a neural network to distinguish between submissions that should be accepted versus those that should be rejected. It's a straightforward supervised learning project that could be built by a graduate student. It could be trained on the last five or ten years worth of accepted and rejected articles with a portion held back for testing.

We might not like what such a NN would say about the process (my guess is that the cover page will turn out to contain some very strong signals) but it could certainly be done.

Posted by: john | Apr 17, 2017 1:32:34 PM

Derek,

(1) The incentive function to generate legal scholarship is to more than just the ability to make things available. You may not personally approve of the incentives structure of legal scholars, but they are what they are.

(2) I think this depends on your standard. A weak screening function is a lot better than none, in my view.

(3) I think copy editing and the like is a pretty standard part of publishing.

(4) I don't see how the ability to print a .pdf solves the archiving problem.

(5) In my experience, the law review editing process always includes lots of substantive engagement with the article's arguments. They are usually the first round of review, in which the editors suggest ways that the article could be significantly improved.

(6) Yes, it does.

(7) That's a possible function, too, although it depends on how much placements actually matter in developing a reputation (something on which opinions differ, I think).

Posted by: Orin Kerr | Apr 17, 2017 12:54:54 PM

I would add to 7:
7. A message to prospective employers about the work one did in law school (a signaling function)
8. The formation of a cohort that works closely together, forming bonds for the future (a networking function)
9. For some, training on how to manage internal and external people, deadlines, budgets, and workflows (a leadership function)

So, contrary to D.C.'s cynical view of the world, there is value even if you only view the members as gunners (which is unlikely in any event - not all gunners get the grades, and not all high achieving students are gunners)

Posted by: Michael Risch | Apr 17, 2017 12:37:15 PM

Orin, thanks for your response. I think a lot of that would have made sense 40 years ago, but given the other technological options we have now the law journal model looks really inefficient. Consider a rather moderate alternative: SSRN + Specialty faculty (or practitioner) blogs that highlight particularly good work. I'm sure it'd be possible for Lexis or West to add these articles the same as anything else.

1. We've got the distribution mechanism. And really it's the search engines, not the journals, which are the distributors. We need ways to electronically search based on topic, not hard copies located in a library.

2. Screening function would be a fair reason, but only if offered only for a few very top journals, and only if we believe students are genuinely capable of separating the wheat from the chaff. The vast bulk of published articles though aren't noteworthy.

3. I agree on this one. Free labor is a nice perk for the professors who get published.

4. You can print PDFs.

5. If we were really concerned about the editorial function, then students on journals would be working with the substantive ideas of the articles rather than toiling away with citation checks. Also, journals would be filled with amicus briefs and white papers, not academic articles.

6. Does this actually happen? Does anyone think more of UCLA if UCLA gets a good article? You probably think more of the author and the author's school rather than the publishing school.

Bonus #7: Build the reputation of the professors getting published. I mean ask yourself why you submit to law journals rather than just putting an article up on SSRN and distributing it through this blog or others. You can easily reach an audience through blogs -- probably far more than you'd reach with just a journal and no blog publicity. If you need help editing, you can probably afford a few hours of research assistant time (if your school isn't already providing it). If you want a permanent version, print it and give it to your library. And you can make your school look good by loudly mentioning them when you post the article to the blog. Seems like the biggest thing missing is that someone unfamiliar with your work (and who lacks the incentive to become familiar) would have no idea how prestigious it is if it's just on SSRN and promoted on Prawfs.

Posted by: Derek Tokaz | Apr 17, 2017 12:27:48 PM

"what function are they intended to serve?"

Initially, to function as a way to disseminate legal scholarship. Now, to pad the CVs of gunner law students, and provide a venue for law faculty to add to their publications list without having to undergo peer review.

Posted by: D.C. | Apr 17, 2017 12:00:19 PM

Depending on what you regard as the complaints, "students" might be a better or additional descriptor of an interest group. It's not just that law reviews want to manage themselves, or keep things as they are. Students generally have an interest in extracurricular activities that allow them to distinguish themselves outside the classroom. So they tend to support maintaining or expanding the number of journals that give meaningful opportunities for participation, which often (but not invariably) end up being student-run. But for others, the number of journals is an important part of the complaints.

Posted by: Ed Swaine | Apr 17, 2017 12:00:13 PM

Derek,

My sense is that law reviews are designed to achieve six things:

1) Encourage legal scholarship by providing a forum and distribution mechanism for its publication (an incentive function).
2) Identify notable scholarship through selecting articles to publish (a screening function).
3) Help edit and polish that scholarship for publication (the editing function -- substantive editing, copy editing, bluebooking, etc.)
4) Provide a permanent version and location of a past published article (so it can be readily found, correctly cited, etc if websites go bust, links rot, or multiple drafts are available).
5) Provide a useful learning experience for students in editing articles and writing student notes (an educational function).
6) Make the law school where the journal is published look good by publishing interesting articles (a reputational function).

I gather, based on your past comments, that you are very dismissive about the ability of law reviews to actually achieve these functions. But that's separate from the goals, which I think are mostly summarized above.

Posted by: Orin Kerr | Apr 17, 2017 11:58:36 AM

Reform is hard because the faculty needed to implement the reform (top law school faculty, however defined) are the ones that benefit the most from the current system. Which leaves us in a position everyone else agrees is terrible

Posted by: Anon | Apr 17, 2017 11:37:08 AM

Should law reviews even exist?

Serious question, what function are they intended to serve? I don't mean the articles. Articles can exist independent of journals. I'm asking what function the journals are meant to serve. Any meaningful discussion on how to fix them is going to have to start with figuring out what they're supposed to do in the first place.

Posted by: Derek Tokaz | Apr 17, 2017 11:32:57 AM

That's excellent news, Brian. I hope other commenters take seriously the potential for reform and offer some good suggestions.

Posted by: Mark Fenster | Apr 17, 2017 11:02:44 AM

Mark, on #4, the AALS Committee on Scholarship has a new project on law review reform. We'll be discussing it, and other ways AALS can contribute to scholarship, at the annual meeting, but very much open to thoughtful comments before then.

Posted by: BDG | Apr 17, 2017 10:57:29 AM

The comments to this entry are closed.