« Web Video and the Academy | Main | A new (?) paper on the Roberts Court »

Tuesday, October 19, 2010

The Great Unranked: Leiter versus the "Third Tier"

It was nice to see Brian Leiter's critique of the US News rankings' "Third Tier" and "Fourth Tier" classification of law schools.  His site's usual focus on "top" schools' scholarly output leaves those of us looking up from below feeling sometimes left out, so much that some go out and do their own studies.

In his post, he makes what strikes me as a quite strong argument against taking the "tiers" in US News too seriously.  The differences between schools within each "Tier" -- from their mission to their teaching to their scholarly output -- are far greater than their similiarities. Yet the classification of certain schools into Tiers has dramatic effect.  Unlike moves in the rankings within the "top 50," the classification into Tiers is essentially a zero-sum game.  If a school in the "Third Tier" moves up into the coveted "ranked" position, that means another school has moved down.

And once you lose this game, you lose your ranking.  My only quibble with Leiter's post is that he refers to schools as "unfairly ranked", when in fact, I think they are "unfairly UNranked."

It's a game with meaningful consequences, at least for law school deans.  When schools slip into the mass of the unranked Third and Fourth Tiers, deans lose their jobs.  Or at least, get apoplectic.  When Buffalo slipped into the Third Tier last year, it's dean was "shocked and dismayed."  (Why someone would be shocked to be treated in an arbitrary fashion by an arbitrary ranking is something I have a hard time understanding, but the dismay can certainly be appreciated). 

When my own school has slipped into the "Third Tier" from its occasional bottom-of-the-second position, students have been disappointed, and those are our customers.  Faculty recruiting may suffer.  Even staff morale may fall. (The main demonstrable effect, however, was a decline in submissions to our law review, since we no longer were in the "Top 100" that some Expresso users select for law review submissions).  As the Washington Monthly noted a few years back in discussing the US News college rankings, "one tiny change in a school's data or in U.S. News' methodology, can bump it from the second 'tier,' where its score is identical to the 51st best school in the country to the third tier where its score is identical to the 176th."

So why does US News keep ranking into "Tiers"?  Two obvious alternatives exist.  One would be to continue ranking schools numerically in the range of, say, 100-160, and leave the remainder in a "schools also worth considering" category.  A few years ago, US News accidentally (?) posted the full positions of all of its law schools when releasing the results on-line.  Bob Morse and crew seem at least superficially aware of the criticism that "Tier"-ing is misleading for students. In discussing the 2011 college rankings, he blogged, "We believe dropping the "Third Tier" would make the rankings less confusing for users."  (By stopping ranking at some point, the magazine, which is obviously interested in readers' feelings, would avoid stigmatizing the bottom-ranked school and its students with the label of "worst" law school in America).

Another alternative would be, ala Forbes' MBA rankings, to only rank the very top (20? 30?) schools, and then leave the rest unranked.  That of course simply changes the trigger point for Tiering (perhaps the schools up higher are more secure in their positions without an arbitrary ranking, but I'm no so sure).  But the main downside for US News would be that fewer ranked schools means fewer potential readers who care.  Incidentally, what percentage of subscribers to the "full access" law rankings on US News.com do you think work as faculty members or deans at ranked schools?

Expanding the rankings would, presumably, increase interest in the rankings, since that would mean that schools solidly stuck in "Tier 3" due to low funding or geographic stigma might now have something to play & hope for.

So why not do it?  Here's my theory.  Once you get to around 60 or 70 in the US News ranking, the differences going further down become miniscule.  That is, the difference between school 90 and school 127 is a hair at most.  Publishing 150 rankings positions would reveal -- to an extent that the current list of 100 obscures -- that the rankings have almost no intellectual coherence.  Moreover, Morse and crew regularly defend their rankings by pointing to stability-- generally, among the ranked schools, there is relatively little wild movement (exceptions, of course, abound, particularly when the methodology changes).  But the tiny gap in the schools between 60 or 70 and 140(?) means that, were all those schools ranked each year, there would be perhaps dozens of moves of more than 20 to 30 positions each year -- something that would make the rankings look as predictable as the PowerBall lottery.


Posted by Geoffrey Rapp on October 19, 2010 at 09:14 AM | Permalink


TrackBack URL for this entry:

Listed below are links to weblogs that reference The Great Unranked: Leiter versus the "Third Tier":


The comments to this entry are closed.