The Unbearable Lightness of Being a U.S. News Voter

The U.S. News rankings are out, and for the first time, I was complicit in the process.  That’s because I controlled not just one but two ballots ranking all schools on their academic reputation.  Every school gets four ballots, and among these, one goes to the most recently tenured faculty member and one to the chair of the school’s faculty appointments committee.  I was both those things for Toledo last year (and I probably will be next year also).  Thus, I got twice the “law porn,” because many schools apparently have separate lists of who occupies these slots, and they don’t cross-reference.

I understand that.  What I don’t understand is how anybody could even try to do this ranking job even semi-competently.  Are voters really expected to read and compare the glossy brochures we get from well over 100 schools?  Go to websites and check out the scholarship of a significant percentage of the faculties of a significant number of schools?  Survey multiple colleagues and friends at other schools as to how they think the University of X compares to Y College of Law in their areas?  Making even a serious attempt at any of this would be a part-time job in itself.

Of course schools ranked in the top 20 have better reputations than schools in the middle of the pack, which have better reputations than schools at the bottom of the rankings.  But the important part is not confirming what we’re all already supposed to know, it’s distinguishing among schools that are reasonably close in the rankings.

What I suspect actually happens is this:

People get positive or negative feelings about individuals in their own field.  They know a few folks outside their field from conferences, the odd article people read in some other field, the occasional guest speaker, and maybe even those they encounter on blogs.  But that is in no way representative.  People also give higher votes, at the margins, to higher ranked or more famous universities (independent of the quality of the law school); places where they have friends, where they or their friends went to school; favored parts of the country, etc.

Also, I suspect there is probably a lot of strategic voting: at the margins:  give lower votes to schools ranked right around where your school is ranked. 

None of these methods would yield results that are, in any overall sense, fair or accurate.  But I can’t think of a way to do this job well.  Suggestions?