Thursday, October 11, 2012
Something I Do Not Like Lots
Hey, I haven't met most of you, and this is crazy, but I want to start my stint on Prawfsblawg with a very mild complaint. My colleague, Michael Froomkin, founded JOTWELL (The Journal of Things We Like Lots), where scholars can write brief essays praising (rather than criticizing) legal scholarship. JOTWELL is truly a wonderful thing, one of many brilliant things Michael has done at Miami. I am truly lucky to have him as a mentor and next door neighbor.
Michael started JOTWELL mainly to fill a gap -- to provide a way for those interested in legal scholarship to find out which new articles and books are worth reading. The Mission Statement notes that "[a] small number of major law journals once served as the gatekeepers of legitimacy and, in so doing, signaled what was important," but now "[g]reat articles appear in relatively obscure places." Consequently, one would think that the vast majority of JOTWELL entries would be about great articles in less-than-famous law journals.
But based on my own experience (and not on any data analysis of any sort), most JOTWELL entries are about articles placed in top law journals. This result (again, based on my own subjective experience) conflicts with the oft-stated complaint that student-run, non-peer reviewed journals do not choose the best articles to publish.
Now, I am not blaming Michael for this development. He is at the mercy of the reviewers who submit review essays to JOTWELL. I also do not blame the reviewers themselves, since the goal of JOTWELL is, after all, to let people "know what we should be reading in related areas," and they may be doing just that. And I simply may be wrong that JOTWELL skews towards articles in top journals. But if I am right, it suggests that maybe the placement crapshoot isn't that much of a crapshoot after all.
UPDATE: I apologize for not mentioning this in the original post. As Michael points out in the comments, I had addressed this concern to Michael, and Michael did tell me that he is planning "to mention this concern in my next annual letter to contributors." Michael, in fact, has been very conscientious in developing JOTWELL, which is why it is the great resource that it is.
Posted by Sergio Campos on October 11, 2012 at 10:44 AM | Permalink
TrackBack URL for this entry:
Listed below are links to weblogs that reference Something I Do Not Like Lots:
Mightn't it instead suggest that maybe the contributors to JOTWELL rely on placement to decide what to write about?
Posted by: anonymous | Oct 11, 2012 10:54:50 AM
As I told you when we discussed this, I had hoped when I started the Jotwell project that more reviews would be of works by less-well-known authors, and planned to mention this concern in my next annual letter to contributors.
Personally I suspect the journal skew comes from people using table of contents services provided by many law libraries. People may tend to read the TOC's of big-name journals more carefully; I suspect more people have the Yale Law Journal routed to them routinely than get the Podunk Trade Dress Law Review. That's certainly true for me, and it has to be a factor as well.
The author skew can have many causes, one of which could be that big-name authors really do deserve their reputations.
There's also another thing to consider, though: many (in some sections, most) of our reviews are of works that are on SSRN which are not placed at all at the time the reviewer writes the review. That may contribute to the author skew, but not the journal skew.
Posted by: Michael Froomkin | Oct 11, 2012 11:19:10 AM
I had the same hunch. The hierarchy of student-edited reviews sorts articles one way or another-- either by directing readers' limited time disproportionately to top journals, or by actually sorting the quality of published work fairly well according to their hierarchy. The latter seems unlikely with student editors, but those who defend the system sometimes speculate that sorting articles by authors' letterhead, which student editors rely on in part, might be a mechanism not much more flawed than its alternatives.
Posted by: Darryl Brown | Oct 11, 2012 11:22:15 AM
For what it's worth, I like reading about the good articles in good places, too, and I love JOTWELL. I also found a great book that I might not have read thanks to a JOTWELL entry. Keep up the great work, Michael and team! (Not that you shouldn't follow through with your plan to highlight more journals...)
Posted by: anon | Oct 11, 2012 12:35:06 PM
The problem with sorting by letterhead, is that law schools in turn sort by journal placement when looking at hiring. It's an ouroboros.
Someone with the necessary expertise needs to actually read and evaluate these articles for the resulting signaling process to have any meaning at all.
Posted by: brad | Oct 11, 2012 5:51:12 PM
For an analysis of what Jotwell reviewed in period (July 2014-2015) see What We Like at http://jotwell.com/what-we-like/. Here's a bit of the conclusion:
"In sum, Jotwell reviews consider authors from throughout the legal academy, both those that are established full professors at top-tier law schools, as well as non-tenured professors at lower-ranked schools and also a small number of students and practitioners. Jotwell additionally brings attention to international scholars and other non-law professors whose work is relevant to legal academics. Finally, we note that in the 2014-2015 year two out of the 194 authors reviewed were reviewed more than once by Jotwell editors.
"Happily, the results presented above seem broadly consistent with the achievement of Jotwell’s twin mission: noting major new contributions in the field and highlighting writing by academics from throughout legal academia. Some work remains to be done to encourage reviewers to think about whether Jotwell’s reviews are proportionally reflective of the racial diversity of legal academia."
Posted by: Michael Froomkin | Oct 29, 2015 10:21:05 PM