Friday, May 20, 2005
Thoughts on Nate's thoughts on the Harvard Law Review selection process
In a comment over at Conglomerate, Nate Oman discusses the Harvard Law Review selection process. It's quite lengthy and involves various stages of review, including a faculty read and a vote by the entire membership of the review.
As Nate notes, the process is more or less an open secret. And as he also notes, Harvard "would pass-up pieces with exploding offers from other journals -- Columbia was particularly fond of this device -- when it would require that we short circuit our process." I'm happy that all of my reads at Columbia were not in vain, and I'm glad that our exploding offer-like policy was useful in keeping pieces with us.
Columbia doesn't actually give exploding offers, but it's pretty close. If Columbia makes an offer following a request for expedited review, the author has one hour to decide whether to accept the offer. This is essentially enough time to hang up and check with Yale and see if they're meeting right now on your piece and can give you an answer within the next hour.
I suspect that CLR's process was largely designed as a defense mechanism against Harvard. After all, one major effect is almost entirely to prevent shopping up to Harvard. This limits the universe of journals-that-we-might-get-shopped-up-to to Yale (Columbia's policy also makes life difficult for Yale, but not impossible). (I don't think that there's much if any incentive to shop to Stanford or Chicago, since it's unclear that they're a step up from Columbia).
And as a Harvard-blocker, the policy seems to be working. Way to go, CLR!
(Questions I'm pondering -- at what point will HLR have to give in to market pressure? Does anyone else employ such a slow process? Is this only feasible at a journal with a lockdown hold on the #1 position?)
TrackBack URL for this entry:
Listed below are links to weblogs that reference Thoughts on Nate's thoughts on the Harvard Law Review selection process:
» A Different Kind of Law Review Competition: from The Volokh Conspiracy
Over at Prawsflawg, Kaimi and several commenters (including myself) are discussing the different strategies top journals use to try to win the competition for ... [Read More]
Tracked on May 20, 2005 4:08:22 PM
» A Different Kind of Law Review Competition: from The Volokh Conspiracy
Over at PrawfsBlawg, Kaimi Wnger and several commenters (including myself) are discussing the different strategies top journals use to try to win the competition for th... [Read More]
Tracked on May 20, 2005 4:13:04 PM
Franz Kafka's parable "Before the Law" in The Trial best sums up the Harvard Law Review article selection process:
"Before the Law stands a doorkeepr. A man from the country comes to this doorkeeper and requests admittance to the Law. But the doorkeeper says that he can't grant him admittance now. The man thinks it over and then asks if he'll be allowed to enter later. 'It's possible,' says the doorkeeper, 'but not now.' . . . [When the man tries to peek beyond the gate] the doorkeeper sees this . . . and says: 'If you're so drawn to it, go ahead and try to enter, even though I've forbidden it. But bear in mind: I'm powerful. And I'm only the lowest doorkeeper. From hall to hall, however, stand doorkeepers each more powerful than the one before. The mere sight of the third is more than even I can bear.'"
Posted by: Daniel Solove | May 20, 2005 12:36:19 PM
Well, I wouldn't be so proud of Columbia's pre-emptory tactics. They are, how shall I put it?, disgraceful. Publication is not a game and should not be treated like one.
Posted by: Anon.Prof. | May 20, 2005 1:17:04 PM
What an odd post. Harvard devotes countless hours to carefully scrutinizing submissions, to researching the submission's originality, and to consulting faculty knowledgeable in the field. Rather then trying to emulate this approach, Columbia tries to exploit it by being less diligent and then using its haste to coerce authors into foregoing an opportunity to publish in a more careful and prestigious journal. Then its editors brag about this in public. And when these editors (and others from journals with similar philosophies) enter the academy, they travel the cocktail party and blog circuit mocking the regime of student edited law reviews whose poor quality they helped to maintain. Strange.
Posted by: Anon | May 20, 2005 1:54:02 PM
How long does the Columbia Law Review give authors to decide on offers not made in the course of expedited review?
Posted by: Orin Kerr | May 20, 2005 1:58:52 PM
Thanks for the feedback, though I obviously don't completely agree with all of the comments. But there are a lot of important points.
1. This isn't a Harvard-versus-Columbia dichotomy as much as it's a Harvard-versus-everyone-else dichotomy. As far as I can tell, everyone except Harvard moves at a faster rate.
At Columbia, we could turn a piece in a few days. Less, if necessary.
2. A commenter says "publication is not a game." That is true. But publication is a competitive process for the journals, as much as it is for the authors. If authors should be free to compete for the best possible journal placements (which is, I think, a generally held belief -- everybody wants to publish in Harvard), then should journals be able to compete for the best possible pieces?
And that's what this is, competition. It stems from a few different sources. First, there is a desire to pick up good pieces, in order to show the value of one's journal. Second, as editors we don't want our efforts to go to waste. As an editor at Columbia, I dedicated time and effort to doing a thorough read. If the piece went to committee, the whole committee dedicated its time and effort to the process. And with that amount of investment, we wouldn't be too thrilled to lose a piece. And so we naturally put a limit on the author's ability to shop us up.
3. Anon#2, everyone does this. Columbia takes articles from NYU. NYU takes articles from Cornell. Cornell takes articles from BU. And so on, down the food chain.
And yes, Harvard takes articles from everyone. As does Yale.
More presitigous journals institute tighter response deadlines as a natural defensive mechanism. (A few years ago, I compiled a chart of response times at top 25 journals. The drop off is noticeable.)
And authors know this. They realize that the price of acceptance at Cornell or Georgetown is a pretty tight dealine. The price of acceptance at Columbia or Stanford is an incredibly tight deadline. For most authors, that's a trade-off they're willing to make, since Columbia and Stanford are both top-5 journals.
4. Anon#2, I think that every top journal dedicates large numbers of hours to evaluating articles. We did a full committee read and preemption check on every piece that we picked up. If there was any question of whether we knew the area of law, we could request a faculty read as well. I can't count the number of classes I missed as a 3L, while on Articles Committee business.
On the other hand, there are distinct differences between the Harvard and Columbia processes:
a. Initial Read.
Every article at Columbia is initially read by one of the seven Articles Editors. An Articles Editor decides whether or not your piece goes to committee.
And AE is one of the prestigious, sought-after board positions. Every AE is a 3L, with a year's law review work behind her and demonstrated excellence at the review. AE's are generally among the top students in the class. (My time at Columbia, most of the AE's were Kent Scholars, Columbia's highest class designation. Several of them had advanced degrees.)
In contrast, at Harvard, the initial cut is done by memo. Any Law Review staffer can be the initial reviewer (including brand-newbie 2L's) and can write a memo that effectively kills your piece.
At Columbia, the Articles Committee has final decisionmaking power. There is nothing like Harvard's full-membership vote.
If you really prefer that every 2L on the review have a chance to weigh in and vote on your piece, that's certainly your prerogative. I don't see why you would want that. The alternative at Columbia is that you've got to get a supermajority of the AE's (who are, you'll recall, generally seven of the top students in their class).
It's much faster to get approval from the AE's. Also, I don't know how much value is added by turning it into a general-body plebescite. The AE's are the experts -- the people who first-read articles, the people who will be editing them. What exactly is gained by letting the general staff vote in a never-ending meeting?
Harvard prepares a preemption memo. Columbia AE's don't operate by memo, but check every piece thoroughly to see if it's been preempted.
d. Faculty Read.
Harvard has a mandatory faculty read. Columbia does not, but regularly requests faculty reads on articles where the editors have questions or concerns.
Advantage: Harvard, perhaps; however, this step adds a lot of time to their process.
The bottom line seems to be that Columbia AE's bust their ass to read every piece that comes through the door, they vote on the pieces as a committee, and they don't have required faculty reads. Harvard AE's are able to farm out much of the screening, and even voting, to the 2L staff. And they take longer.
How exactly is this process "less diligent"?
5. Orin, Columbia gives 24 hours for pieces not picked up on expedite.
Because of the sheer numbers of expedites, something like 80% of the pieces my year were picked up on expedite.
6. The net result is this. Dan Markel gets a piece picked up at a fast-deadline place like Cornell. He calls up top journals and says "I have 24 hours to get back to Cornell." It's 11:00 am.
The EAE calls me and says "Kaimi, I need a read of Markel's piece right now." I ditch Admin and read Markel's piece. It's now 2:00 pm.
I like it, so I recommend full committee read. The other committee members start their reads. I do research on preemption and pull any relevant articles. If the piece needs a faculty read, we ask for one. (We're unlikely to get one in 24 hours).
We meet the next morning as a committee at 9 am. We discuss the piece, its strengths, weaknesses, etc. We discuss any preemption issues, and anything that our faculty reader (if we needed / got one) said. We vote.
At 11 am (Dan's deadline) we make him an offer.
Meanwhile, Dan also called Harvard with his 24-hour deadline. They assigned his piece to a 2L to prescreen and write a memo on. That 2L wrote a memo, and that memo then went to committee, who read the memo and decided the wanted to further discuss the piece. They send the article out for a faculty read. There is talk about having a general membership meeting on Markel. At some point in this process, the 24-hour deadline arrives. Dan does not get an offer from Harvard.
Posted by: Kaimi | May 20, 2005 3:14:09 PM
A note: When I was at HLR we dispensed with the initial memo for pieces with expediated reviews, which were considered by the articles committee.
One advantage in theory of the including all editors in the final decision making process is that it prevents the content of the review from being unduly skewed by the personal interests of the articles committee. On the other hand, my personal opinion is that the intellectual interests of the articles committee at HLR were probably wider than the intellectual interests of the median HLR editor, which may be one of the reasons that HLR is so skewed in favor of constitutional and federal law (unfortunately in my humble opinion).
Posted by: Nate Oman | May 20, 2005 4:17:59 PM
That makes sense.
In reality, I can see instances where the articles committee alone is less effective. Particularly if it's a dysfunctional group, where Editor A hates Editor B and will never vote for anything Editor B likes.
Or where the committee is biased in some way, with too many people espousing one set ideology and that prevents good review of other papers.
(That wasn't my experience, but it should be considered as a potential drawback to committee only review.)
On the other hand, full-body review potentially leads to homogenization towards certain popular topics.
Posted by: Kaimi | May 20, 2005 4:28:15 PM
A reader from any other academic discipine would bust a gut laughing to read this thread, with its description of an articles selection process that sounds more like something that takes place in the trading pits at the Chicago Mercantile Exchange. In other areas, articles are chosen after careful review by experts in the area. Exploding offers, fast-track deadlines, expedited reviews, rushed committee meetings, and haggling over Harvard vs. Columbia are all artifacts of our bizarre system of multiple submissions to student-edited journals.
I know it's hopeless to argue for true peer review -- we profs wouldn't want the additional work -- so student involvement in the process is here to stay. But I've got to believe that the process would be a little less frenzied if it were possible to cut down on the volume of submissions that each journal receives. Over at Legal Ethics Forum, John Steele and I discussed one voluntary, opt-in proposal to reduce the flood of submissions. (I don't know how to make hyperlinks in comments -- I'm a blog newbie -- but the link is http://legalethicsforum.typepad.com/blog/2005/03/ethical_issues_.html.) If most authors submitted to only one of Harvard, Yale, Columbia, Penn, Michigan, or the other top journals, the process might be a bit more deliberative. And maybe if authors spread their submissions around, there would be less of a winner-take-all dynamic, in which people believe that a Harvard placement is really somehow better than Columbia or Penn.
It's true that peer review selection in other areas has costs. It takes forever to get an article accepted, for one thing, and the journals can also be clubby and hostile to ideas that are outside the mainstream. We don't have to eliminate multiple simultaneous submissions to law reviews entirely, just cut back the volume of submissions per manuscript to something more reasonable.
Posted by: Brad Wendel | May 20, 2005 4:56:35 PM
A few points in response to your comparison between the CLR and HLR review processes (based on information that I admit is a few years old, and might not reflect current practices at the HLR):
1. I find it difficult to see how any law review articles committee could read all of its submissions as thoroughly as Harvard does through its practice of assigning 2Ls (and some 3Ls) to write a memo on each submission. Law reviews receive hundreds or thousands of submissions each year, and a committee of six or seven people cannot possibly conduct initial sorting screens as thoroughly as a group of 40 or so editors. Harvard's process is also anonymous, while I suspect that articles committees at many other journals use the author's name or institutional affiliation as a proxy for winnowing or prioritizing the stack of unread submissions. The combination of a thorough memo by a 2L or 3L coupled with review of that memo by a member of the articles committee, and if necessary further review of the submission itself if there is any doubt about the initial screener's negative review, seems more desirable than expecting an articles committee to screen all submissions by itself. Also, the HLR's screening process frees up the HLR articles committee to focus most of its time on the few hundred most plausible submissions rather than the more mountainous haystack of weaker submissions.
2. At the HLR, articles are considered by the full membership only after approval by the full articles committee (which, like its CLR counterpart, is comprised of top students and often includes editors with advanced degrees. Many of the committee's alumni are now academics). To be accepted, an article thus must win approval both from a narrow group of articles-selection-specialists and from a broader group with more diverse interests. I can see this two-stage process cutting both ways: sometimes the general membership's added scrutiny reveals issues that the articles committee may have missed, and sometimes the membership needlessly kills pieces that the committee rightly favored.
3. Commentators seem to assume that Harvard is slower than other journals. This is sometimes true, but not always. In the right circumstances, the articles committee's read, the originality check, and the faculty read can occur simultaneously, and the O-read can occur soon thereafter. I am aware of many cases where articles have been accepted within 48 hours of the first person reading them, although this is obviously not the norm.
4. The law review selection process seems mired in a serious collective action problem: some journals apparently would like to consult faculty, but fear that doing so routinely will lead other, less diligent, journals to poach submissions through short-fuse exploding offers. It would be nice if all the top journals could agree to submit all the pieces at the final stage of the review process to faculty for review -- and if faculty members would agree to actually read them when asked. The top law reviews have already banded together to encourage submissions to be shorter -- maybe a faculty-read initiative could be their next effort.
Posted by: Anon | May 20, 2005 5:21:17 PM
1. Agreed that faculty reads all around would be a good addition to the mix. I don't know how this could be negotiated, but I suspect that there's a way somehow.
2. I'm sorry if my last comment was a bit combative. I felt a little defensive after a process that involved me personally investing hundreds of hours of time and all sorts of energy (and watching my 3L grades go to hell) was characterized as "less diligent."
I'm glad to engage in friendly rivalry, but I do think that the HLR method has its uses. However, I think that the current competitive environment may mean that HLR must ultimately adapt to changed times.
3. You're right that there are disadvantages to having an AE look at everything, and the main disadvantage is time. Columbia gets upwards of 2500 pieces a year. Divide by seven, and you've got 400 articles per AE.
There are a few things to note:
a. We read every article. We _don't_ read every article cover-to-cover. By the time an AE has a month on the job, she's got a keen sense for which articles can be killed quickly. These die for one of a few reasons:
-Scope. If the scope is such that the article cannot meet CLR standards, it dies fast. For example, one article I reviewed about some recent changes in the regulation affecting livestock feed. Even the best possible article on that topic is not something CLR is going to publish. An AE can often kill an article on scope grounds within the first 5 pages.
-Writing. There are many bad, bad writers. If the first 5 pages are unreadable, there's no need to keep going.
-All manner of wacky gimmicks (you'd be amazed what people come up with). The article with the (bad) hand-drawn illustrations? Gone. The article which is (per its abstract) entirely based on the smoke and mirrors of a non-falsifiable assertion? Gone. The 400-page article? (Yes, we got one of those). Gone. We're not going to dedicate 400 pages to you unless you're Oliver Wendell Holmes.
The end result was that at least a third of the submissions were gone by page 10.
b. We ended up relying on expedites, probably quite a bit more than Harvard. We picked up something like 80% of our pieces on expedite. So in a sense, we were farming out work too -- farming it out to NYU and Penn and Cornell.
c. We used other proxies to prioritize submissions. No one got rejected because of lack of clout. But if a Cass Sunstein paper and a Kaimi Wenger paper came into the office on the same day, the Sunstein would get read first.
Posted by: Kaimi | May 20, 2005 5:53:22 PM
Brad brings up an interesting if oft beaten topic. I think that legal academia would benefit from more peer review in the journals but not dramatically benefit. Peer review is supposed to provide an independent signal of an articles value, but since the primary audience of the articles is those who are experts and therefore ought to be able to access the merits of a piece without the signal. Hence it is not clear that the academic community as an academic community benefits tremendously from peer review since they are both (1) the primary consumers of their own product; and, (2) the people who are least likely to to need others to vouch for the quality of a particular piece of work.
Where peer review would be valuable is in the context of interdisciplinary work, where the consumers are only likely to be experts in one of the disciplines involved in the study. Here, it seems that we have two problems in moving to a peer review system:
1. Law professors are too lazy to work as editors and peer reviewers.
2. (And more importantly IMHO), peer reviewed pieces don't have greater institutional value. In other words, if one suffers through the much slower production process at a peer review journal and produces a single article rather than two pieces in the law reviews you are unlikely to be rewarded at tenure review time.
What is interesting, of course, is that it makes sense for peer review not to count at tenure review time. The reason being is that one's work is presumeably judged at that point on the merits by fellow experts in the field. They don't need the signal of peer review and there is no reason for them to place extra value on it.
Posted by: Nate Oman | May 20, 2005 6:41:41 PM
But perhaps it makes sense for them to place value on it if it's outside of their field. If I'm a legal philosopher and I publish in Legal Theory, then when I'm up for tenure the con law person and the tax person and the property person on the tenure committee, who don't know the scholarship of legal philosophy, can use that signal (that I was able to convince existing legal philosophers to publish me) in their decision.
(I don't know that things really work that way in practice, but it seems like a good way that they _could_ work, in theory).
Posted by: Kaimi | May 20, 2005 6:50:46 PM
Re Screening out obviously inappropriate submissions early: I have always pitied the poor 2L editor of the HLR who gets the article on livestock feed, knows it has zero chance of being accepted, and yet feels pressure to write a decent memo on it to impress the articles committee members whose assessment of the 2L might matter down the road. In that respect, the HLR selection process exploits its labor pool and is wasteful. The problem is that in practice, aside from a few obvious factors, it is very difficult to develop criteria to sort the obviously unpublishable from the potentially publishable without reading at least the first part of the article, and sometimes 2L screeners who carefully read an ostensibly weak article discover a hidden gem. Which leads to a different question: why are the authors of articles about livestock feed wasting their time, and editors' time, sending their pieces to the HLR and CLR? Maybe each law review should charge a $5 submission fee (collectible on-line at the same time a draft is submitted on-line): high enough to potentially deter authors from sending a draft to 30-40 journals that will never accept it, but low enough not to deter authors from sending out articles that have a decent chance of being accepted. Or, to take the idea further, perhaps reviews could adopt a sliding scale of fees: $20 base; lowered to $10 if the author promises to have the article considered by no more than 10 journals at any one time; and no fee if the author promises to have the article considered by no more than five journals at one time. Enforcement might be difficult, but the honor system should work.
Posted by: anon | May 21, 2005 12:59:59 AM
Here's a modest proposal. Publication, as Kaimi states, is a competition; if there's a way to gain an advantage over your rivals, go for it, even if you have to dash some hopes and expectations along the way! So why, I wonder, are we authors playing with one hand behind our backs?
Say you get a call from a place like Exploding Offer Law Review, which gives you an hour (or 24). Why not graciously accept the offer before the deadline arrives -- and then continue shopping the piece at your favorite publication. If Yale takes it, call up Exploding and retact your acceptance.
To be sure, most authors, as repeat players, will shy away from doing this, for fear of being put on a law review blacklist. But some of us (tenured folks at least) out to risk taking one for the team. After all, as a matter of tactics in this Hobbsean war between law students and their professors, this approach might just be the way to defuse the exploding offer.
Now watch our student editors suddenly turn sanctimonious: such a move, they would assert, is utterly unethical. But why is that? This is a game, a competition; the move I outline is no more problematic than the intentional short circuiting of the Harvard process that gave rise to it. And legally speaking, of course, publication agreements are nothing more than contracts. Given the surfeit of beautifully footnoted pieces out there, the Exploding Law Review will suffer no damages. Can you say efficient breach?
Law Profs of the world, unite! You have nothing to lose but your exploding offers.
Posted by: Anon. Prof. | May 21, 2005 2:32:45 AM
Not to get into an entirely different discussion but, as a factual matter, NYU gets expidite requests from Columbia-accepted articles in a number of fields, and this trend has increased steadily over the past few years. I readily acknowledge that Columbia recieves many expidites from NYU as well, but it is by no means a one-way street.
Posted by: AnonEditor | May 21, 2005 2:58:08 AM
Anon. Prof.: In order for you to be certain that the breach is efficient the student editors would need to be able to recover the full value of their expectation from the professor. (Setting aside for a moment the question of whether this would cause the editors to take inefficiently low levels of precautionary measures to guard against professorial breach.)
Posted by: Nate Oman | May 23, 2005 10:21:58 AM
You provide a great example of why "efficient breach" is a joke of a theory. If the editors of the Exploding Law Review invest time reading your piece because you have agreed to the rules of the game (i.e., if you get an offer, you get a limited time to accept, and your acceptance would be in good faith if given), your breach is highly inefficient.
In reliance on your statements, they have dropped everything to read your piece (as opposed to reading other pieces or doing non-Law Review work). But it turns out that you (hypothetically) have no honor, and your word is worthless.
In addition to the wasted effort, it's also possible that the editors forwent other available articles (perhaps on similar topics, or just at the top of the bin) while rushing to read yours, meaning that these other articles were lost to competing journals. How can these losses be evaluated? Of course they cannot be measured in money, so the courts will largely ignore them. Score another point for the morality of honoring ones contracts as a pillar holding up our economy.
As always in the fantasy land of efficient breach, someone must ask, "Efficient for whom?"
Posted by: Fmr. AE | May 24, 2005 5:46:11 PM