Deadwood Reports and the Choices They Make

I was out celebrating my second anniversary(!) with my wife over lunch, so I'm a little late by blogospheric standards to report this, but in case you haven't seen it, the venerable Green Bag will be issuing an annual Deadwood Report. (H/t: Robin Craig; more coverage by Brian Leiter, Orin Kerr, and Paul Secunda, who proposes bobblehead dolls made in honor of law school deans and various prawfs).

After the jump, I excerpt parts of the story in Inside Higher Ed that gives the gist of this forthcoming report. To my mind, the most vivid and important choice discussed in Ross Davies' piece on SSRN is the decision to not count (at all!) any scholarship published by faculty in a law journal edited by that law school's students or by a faculty colleague who serves as an editor. I think this is a salutary development for a few reasons, but I think it's easier to justify vis-a-vis the student journals than the faculty ones. So it's clear, if Harvard Law faculty publish in the Harvard Law Review, that will count against the school on the deadwood report. Same thing if Harvard Law faculty publish in the new Journal of Legal Analysis, edited by Shavell and Ramseyer. And of course, JLS is now off-limits, in this respect, to all the UChicago faculty.

My sense is that this decision to exclude home-seeded publications would be controversial at some faculties, especially those at the top few law schools, if and only if those faculty members were sensitive to the signals these rankings decisions emit.  But because most faculty already at Harvard or Yale could care less about how the market perceives their decisions -- after all, they're already at Yale or Harvard -- it probably won't lead to a speech by Harold or Elena saying: hey everybody, don't publish in the home journal anymore. Ross Davies' minions won't like it.

The rest of the market outside the top 4 or 5 schools, however, is likely to internalize that message and so we'll see less "lock-up" of the top (and all the other) law reviews by folks with a home-school advantage. This seems to me to be a very good development with respect to the student edited law reviews. In theory, the faculty-edited law reviews that blindly assess incoming submissions should be excused from Ross' rule. But perhaps there is true skepticism about whether blind review can occur even in faculty journals. I haven't been in the business long enough to have a strong view about this--sometimes the faculty edited journals seem more driven by informal networks than what's published in the law reviews, but sometimes it's not like that at all.  And of course, if blind review really does exist at a law review, say like Harvard's, then HLS faculty shouldn't be penalized for publishing there.  (My soft recollection from my experience at Harvard Law Review is that all manuscripts submitted were read anonymously by a member of the Board of Editors, but that HLS faculty members still had a leg up in the process; that said, we did turn down submissions by HLS faculty and those decisions were more fraught than other ones. Perhaps things have changed.)

Anyway, my questions for you all are: first, is there anything more to be said against Ross' rule? Should there be exceptions for schools whose journals (whether faculty or student-edited) follow a blind review process for submissions? What other things do you think should be weighed in the Deadwood Reports that aren't already mentioned? Last, is Paul Secunda correct about the untapped market for prawf-bobblehead dolls :-)

Update: One of my favorite readers of this site wrote in to ask whether blind review of law articles is possible when so many people know what's going on with respect to conference papers and SSRN/bepress drafts. To the extent that's true, it cuts against any exceptions to the Green Bag Rule.

Starting this spring, it will begin work on the “Deadwood Report,” which it envisions being an annual assessment of “whether faculty members do the work that the law schools say they do.” The journal acknowledges that the ranking will provide “rough and admittedly partial” measures of law school faculty quality, but posits that by being transparent (it will disclose the sources of its data and how it derives its numbers and rankings from those data), and by bringing more information into public view, “it will help law school applicants make better decisions about where to study or work.... We are trying to do some good here.” (The editors have an ulterior motive, too: compelling law schools to make public better information about their operations — more on that later.)

What exactly will the Deadwood Report measure? Law schools, the editors write, “generally hold themselves out as institutions led by faculties whose members are committed to teaching, scholarship, and service.” They argue that the best teachers tend to be active scholars and vice versa, “and all the best lawyers of every stripe engage in service for the public good.... Evidence of the law schools’ commitment to this view is reflected in the practically universal requirement of high achievement in all three areas for tenure. And so we should be able to say with some confidence that a good law school will have a faculty consisting of hard-working teacher-scholar-humanitarians,” the Green Bag editorial says.

“The Deadwood Report will simply test the accuracy of that picture,” the journal’s editors write. “Our focus will be on the most dully objective of measures: whether the work is being done — whether each law school faculty member is teaching courses, publishing scholarly works, and performing pro bono service.” (The journal plans to start with teaching and research, turning only eventually to service, and notes that it does not plan — “at least not yet” — to answer what it calls the “trickier and more entertaining subjective questions: whether the teaching is effective, whether the scholarship is sound, whether the service is in the public interest.")

The Green Bag’s method for answering those questions sounds like it will be painstaking. Its staff (editor Ross E. Davies and research assistants) plan to:

• Download a law school’s faculty Web pages, course catalogs, and publications lists.
• Compile the data, with an emphasis on recent scholarship and recent teaching ("A school whose faculty is heavy with people who used to be active might do well in a citation or reputation study, but it will do poorly in the Deadwood Report").
• Analyze the data, using a still-to-be-finalized process of “sorting and weighting.” Basic principles: “We are interested in well-rounded, activie faculty members, and so we will give more weight to the moderately active teacher-writer than to the hyper-writer who neglects teaching or to the hyper-teacher who neglects writing.” Schools will also be rewarded for having well-rounded faculties, rather than a handful who are big-time scholars and a bunch of others who aren’t.
• Send each school’s dean his or her institution’s preliminary results for a chance to correct inaccuracies.
• Correct the errors and publish the results.

The journal’s editors offer some advice to law school deans, which offers additional evidence about their motives in joining the rankings game. Keep your Web sites up to date, since that is where all of the rankings’ information will come from. “This seems reasonable to us because your Web site is surely where most applicants and other inquisitive people go for information about your law school. If a school cannot be bothered to provide accurate information about the teaching, scholarship and service of its own faculty on its own Web site, it deserves to be haunted by any inaccuracies.”

“Puffery is double-edged,” the Green Bag warns. If a law school’s “faculty” page offers a long list of names, the journal’s editors will include and assess them all in the school’s “deadwood” numbers. “Inflated denominators will not be helpful to you,” the journal’s editorial says. “If you have employees who are employed to teach but not to write, or to write but not to teach,” or who once did one or both but no longer do, or who are on leave, “you might be well-served — and people visiting your Web site would certainly be better-informed — if you moved those folks off your list of “Faculty” and onto lists labeled, perhaps, “Instructors” and “Researchers” and “Emeriti” and “Administrators” and “On Leave.” (Visiting instructors will be treated differently.)

While they generally swear off assessments of quality, the editors make clear that they will make some judgments. “Teaching Property or Torts or Individual Tax to an auditorium filled with students is not the same as co-teaching a half-semester seminar on a highly specialized topic with three colleagues, a weekly guest speaker, and enrollment limited to 12. Do not expect us to give them the same weight.” And “we will be taking account of scholarly books and articles in scholarly journals. Not novels. Not editorials, even if they appear in The New York Times or The Wall Street Journal,” unless “we find on your Web site official regulations indicating that for tenure purposes your law school equates works of fiction, letters to the editor, and the like with conventional works of scholarship, and if further inquiry reveals that your school has actually awarded tenure on the basis of such publications...”

And, potentially controversially: “Works appearing in organs published by your school or your students, or on which a member of your faculty serves as an editor or in some similar capacity, do not count. The pressure to make publication decisions on grounds other than scholarly merit is too great, especially when relationships between students and teachers ... are in play.” ("In that same spirit,” the editors note, “we will not count anything published by the Green Bag, not because we do not publish scholarly works, but because we are wed to Caesar for this project.")