« A Reply to Anonymous Commenter Number 3,497 | Main | Talkin' 'Bout Our Generation: the Rise of Anti-Corruption Law »

Thursday, September 08, 2011

Old Question: How Do You "Rank" a Specialty Journal? New Question: How Do You Rank an Online Law Review?

A frequent question that gets asked on the blogosphere and in law school offices is the value of publishing in a specialty journal vs. publishing in a flagship law review. And the answers always seem to be across the board about how to "rank" a specialty journal. I've seen/heard the following and more: (1) Add 25 to the rank of the law school to get the rank of the specialty journal; (2) Add 50 to the rank of the law school; (3) Add 100 to the rank of the law school; (4) The top 3 specialty journals in a given field are competitive with "top" law reviews, but there's a big drop off after that; (5) It's the top 5 specialty journals in a given field; (5) It depends on the field; specialty journals are more respected in certain fields (e.g., international law), and less respected in others; (6) See where the particular specialty journal ranks in the W&L rankings; (7) See who is publishing in the specialty journal; and (8) Damn the rankings; if you want people in that field to read the article, publish it in the specialty journal.

I think we've now reached the point where we can add another question to the mix: What's the value of publishing in an online law review? These online journals are a relatively new development in legal academia, but the number of them has exploded in the last few years. I publish an annual Submission Guide for Online Law Review Supplements, and, by my latest count, there are such online journals at 7 out of the top 10 and 21 (soon to be 23) out of the top 50 law schools (by USNWR rank).

Moreover, these online journals are starting to do very well in the W&L rankings, and I think this makes sense because they allow authors to get the first word (well, second after blogs) on hot legal issues, meaning that they are likely to be cited. The Northwestern University Law Review Colloquy comes in at #130 in the combined ranking of general and specialty journals, tied with the University of Richmond Law Review and ahead of 4 out of the 5 specialty journals at Northwestern. And Virginia Law Review In Brief comes in at #197, tied with the Denver University Law Review and ahead of 6 out of the 8 specialty journals at Virginia. Also, it's important to keep in mind that the current W&L rankings track citations from 2003-2010, and most online journals haven't been around that long (The Colloquy started in 2006 and In Brief started in 2007), so it's likely that these journals are underranked.

Now, some online journals only publish very short pieces or responses to articles, so an author isn't faced with a a choice between an online journal and flagship law reviews and specialty journals. But other online journals publish longer articles. For instance, Cardozo Law Revew's de•novo publishes pieces up to 25 pages. George Washington Law Review's Arguendo publishes pieces up to 10,000 words. An author easily could have an offer from one of these online journals and also have an offer from flagship law reviews and specialty journals.

So, here's my question: If you are advising a prospective law professor or a pretenure law professor who has the choice to publish in an online journal or a more traditional law review, do you give the author the same advice that you give with regard to specialty journals? If you tell authors to add 50 to the rank of a law school to get a specialty journal's rank, do you do the same with online journals? If the W&L rankings are good enough for specialty journals, are they good enough for online journals? Or does your advice differ, and if so, why?

-Colin Miller

Posted by Evidence ProfBlogger on September 8, 2011 at 09:57 AM | Permalink

TrackBack

TrackBack URL for this entry:
https://www.typepad.com/services/trackback/6a00d8341c6a7953ef0153916dbd40970b

Listed below are links to weblogs that reference Old Question: How Do You "Rank" a Specialty Journal? New Question: How Do You Rank an Online Law Review?:

Comments

So just to recap: we can't be bothered to read scholarship, so we judge its quality based on placement; we can't be bothered to read the rankings that, albeit imperfectly, judge the quality of those placements; instead, we rely on an even more deeply flawed ranking system that does't even purport to measure the right proxy.

Posted by: anon | Sep 18, 2011 2:51:29 PM

Most people do not bother to look up the Washington and Lee rankings. It is the law school's ranking that is the most important for main as well as specialty journals. I don't think anyone would see an offer from a Harvard or Yale specialty journal as equal to an offer from a tier 3 or 4 school. Adding 50 is probably closer to how the journal is seen by most people, but I think even this number goes down as you get to specialty journals at T14 schools.

Posted by: juniorprof | Sep 18, 2011 1:17:35 PM

Rule of thumb might be good for a few reasons: (1) some might want W&L to be one of several factors rather than the single determinative factor; (2) even if one wants W&L to be determinative, sometimes he or she won't be able to access (or won't want the trouble of accessing) W&L when evaluating journals; (3) the W&L rankings shift around quite often, especially after the top 100, so the rule of thumb might be good for evaluating older publications. Those are just a few off the top of my head.

Of course, the best way to evaluate pieces is to read them -- place of publication on student-run journals is a very poor proxy for quality. It's easy to forget that.

Posted by: Young Prof | Sep 18, 2011 12:14:01 PM

If we have the W&L rankings, why do we need a rule of thumb?

Posted by: anon | Sep 18, 2011 11:16:59 AM

This is hardly scientific, but if you believe that the Washington and Lee Rankings ought to be a considerable factor in developing a rule of thumb for specialty law journal ranking, then you might find the numbers given thus far to be considerably more favorable to specialty journals than they ought to be. I looked at a fairly random sample of top schools (I took schools ranked at 1, 10, 20, 30, 40 and 50), and for those schools, I identified both where their flagship law review is ranked on W&L and then where its top specialty journal is ranked on W&L.

This is what I found (considering student edited journals only): [School: Flagship Rank, Top Specialty Rank] Yale: 3, 58; Duke: 21, 164; GW: 36, 210; Emory: 26, 288; George Mason: 170, 378; American: 47, 118.

Even if we ignore that this methodology is arguably generous to specialty journals--it only counts their highest ranked specialty journal--the rule of thumb is perhaps more like "add 150."

Posted by: Young Prof | Sep 18, 2011 12:42:09 AM

This question reminds me of the time I asked my former dean (not at my current school, or even the school I just left) about a choice between Temple's law review and one of Yale's specialty journals; he said it was a tossup. So that sounds like an "add 50" rule.

Posted by: Michael Lewyn | Sep 17, 2011 9:12:43 PM

I must say that I find these discussions to be very disturbing, similar to our obsessions with rankings in virtually all other aspects of legal education.

I won't try to summarize all my objections in this short space. In the interest of boosting my SSRN rankings, however, I invite anyone who wants a different perspective to look at a recent short piece I wrote, "Therapeutic Jurisprudence and the Practice of Legal Scholarship," here:

http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1585311

I promise that this is not simply a rant. Rather, it takes a look at our obsession with prestige & rankings and suggests a healthier approach to legal scholarship.

Sincerely,
David Yamada
Suffolk University Law School, Boston

Posted by: David Yamada | Sep 15, 2011 2:43:37 PM

Keep in mind that there are other things that differ between journals besides just their signaling value. On-line journals will differ from others on such factors as the time to publication, the attention one gets from the editorial staff (given the turn-around time for the on-line journal, likely less), and perhaps in the likelihood that editorial attention will represent value added (e.g., if you are comparing the staff at the Yale Law Journal to the staff at a law school where students are not typically as skilled at legal writing).

For these reasons I've favored on-line journals where I have a piece that is topical, aimed for a broader audience, and not as theoretically complex as a more traditional academic-type piece. That plays to their strengths: it's published while still relevant, you get good style editing advice, and you're not so worried about not having three or four chances to revise.

Posted by: BDG | Sep 9, 2011 9:03:33 AM

On-line supplements are great placements for shorter/essay-length pieces, especially the supplements for the top-10 journals. I would encourage untenured people to place essays there, on the assumption that this is not all that they are doing and that there is other writing getting into print journals. Remember, as well, that some of the on-line journals will publish or consider publishing on-line pieces in the book; that latter republication plays well with P&T and appointments committees.

As for specialty journals, with secondary journals at top-15 schools, I have played by the "+50 rule"--take the school ranking and add 50 for a general sense. Now, some secondary journals are better than this and are essentially top-50 placements by themselves (Harvard CR-CL, for example). But outside that, this rough rule works pretty well.

Posted by: Howard Wasserman | Sep 8, 2011 9:57:47 PM

It's true that the best way to determine the quality of an article is to read it. But I doubt that anyone (including members of tenure committees) can evaluate an article entirely objectively without being influenced (even if only subconsciously) by the journal in which it is published. (Lots of psychology research over the last couple of decades has been dedicated to showing over and over again that we are not nearly as rational and objective as we subjectively believe ourselves to be. See e.g., Ziva Kunda, The Case for Motivated Reasoning, 108 Psychological Bulletin 480 (1990).) Thus I would guess that an article that placed in the Harvard Law Review would be ranked better by almost everyone than that same article if it placed in a third tier law review.

Posted by: Stuart Ford | Sep 8, 2011 5:38:40 PM

andy, that's definitely a fair point, and the hope would certainly be that the tenure/hiring committee would actually read the article and make their own judgments about its quality. Whether that actually happens is another question.

Let's say that you're a prospective law professor. Like many prospective law professors nowadays, you've already published 1, 2, or 3 articles. You've now written an article that you plan to use as your job talk. You've consciously chosen to make this article on the short(er) side (15-25 pages) so that it is easier to present as a job talk (which I think a smart idea). The article deals with international law. You have offers from:

-Cardozo Law Revew's de•novo (school rank=#50; W&L journal rank=353);

-The Florida Journal of International Law; and (school rank=#47; W&L journal rank=353); and

-The Tulane Journal of International and Comparative Law (school rank=#47; W&L journal rank=353).

Now, it would be nice if every law professor deciding whether to hire you has read this article. But, some professors will not read any of your articles. Some will read 1 or more but not all of your articles. Given this, it's probably safe to assume that a minority of professors will read this particular article. So, with regard to the majority of professors, which journal looks better? I would guess that the answer would be to take the offer from Florida or Tulane over the offer from Cardozo even though the #s look about the same and even though de•novo is likely to rise in the W&L journal rankings given that it started in 2009 and the W&L rankings track citations from 2003-2010. And that's probably the advice would give. In a few years, though, I think that my advice might be different.

Posted by: Colin Miller | Sep 8, 2011 5:20:36 PM

when in doubt about the journal in which an article appears (the most important thing), it's always possible to, as a secondary measure, actually read the article to determine quality.

Posted by: andy | Sep 8, 2011 4:37:34 PM

Related: how do peer-reviewed speciality journals compare? How are they viewed by hiring and tenure committees? Is there general awareness in the legal academy as to what journals are peer-reviewed?

Posted by: anon | Sep 8, 2011 2:05:43 PM

Doug - publication in a journal is critical and most entry-level candidates these days for tenure track positions have at least one and sometimes three or four publications. Lots of things get put on SSRN, not all of them very good. And SSRN plays no gatekeeping role, all you need to put something on it is an email address. There are obviously problems with treating publication in a particular journal as a proxy for quality (including the letterhead effect), but it at least shows that five or six people (the editorial board at the journal) reviewed the text and thought it made sense and contributed something interesting to the literature.

Posted by: Stuart Ford | Sep 8, 2011 11:01:32 AM

Let me say first that I have absolutely no data to back up my conclusions on this point. It is entirely intuition and the inherent conservatism of the pre-tenured professor contemplating evaluation by a tenure committee. But I have been very reluctant to publish in online supplements because of concern that the tenure committee will not accord them equal weight to more traditional law reviews (even specialty journals).

Once enough time has passed for us to see whether these online supplements will really contribute to the scholarship (and be cited) equivalently to law reviews, then there will be less risk in publishing there. Colin's statistics suggest that that time might be fast approaching. But for now the risk of being an early adopter is too great for me to be comfortable with it.

Posted by: Stuart Ford | Sep 8, 2011 10:55:49 AM

I have another question for the professor. Is it sufficient for the un-tenured and/or job seeking scholar to post on SSRN or is it necessary to be published in a journal somewhere?

Posted by: Doug Levene | Sep 8, 2011 10:11:12 AM

The comments to this entry are closed.