What are we voting about in US News?

The element of the US News rankings which is far and away the most heavily weighted (25%) is the school's quality assessment from law professors. Right now, it's unclear what criteria faculty use individually, and in the aggregate, the assessment simply replicates previous US News rankings, as Indiana's Jeffrey Stake has shown. We can do better than that.

So as a future (I hope) US News voter, I’m trying to figure out what exactly I’m voting about – and what audience I’m voting for -- in assessing the quality of a school’s “program,” as U.S. News appears to instruct.

I guess the way the rankings work is this: they're designed to help employers figure out from where to hire, and then the idea is that students would use the rankings derivatively -- that is, not as an actual sign of the quality of the school, but as a sign of what employers will think of the quality of the school and therefore how it will affect their career prospects. Russell Korobkin of UCLA, whose 1998 essay on rankings led in part to him giving the keynote address -- framed as a response to Cass Sunstein and Richard Posner's contributions -- at the excellent Indiana symposium on rankings a few years ago, has called this the "primary purpose" of rankings: to "coordinate the placement of law students with legal employers." This seems right to me.

If that's the case, then when I fill out the US News survey as (again, I hope) a newly tenured professor at the University of Georgia in a few years, I’m going to be evaluating each school's "program" for future employers directly, and indirectly for prospective students.

So what do employers want to know, and how can I help?

Let's look at the whole picture. Overall, U.S. News basically provides information in four categories (as described by them): quality assessment, student selectivity, faculty resources, and placement success. The survey we fill out is most of the quality assessment piece, along with the practitioner survey. Of the rest, student selectivity must be in there to give employers some rough sense of the graduates' abilities before law school, placement success could be either designed to give new employers a signal about student ability or to give students a sense of how they're going to do in the market. Then there's faculty resources, which most think is noise or worse, but must be designed to get at some measure of the quality of the education itself.

So given what employers and prospective students are told from the other three categories, what "program" ought we be assessing when doing surveys? I think the choices are: our program of knowledge production, lawyer education, or both. Well, what would employers and prospective students want to know? Certainly, if employers already have some sense of student abilities coming in from the numerical credentials, they would probably want to know, as Nancy Rapoport has put it, about the "value added" that the particular school provides the future lawyers they're considering hiring. They want to know about the quality of the educational program, not knowledge production.

I can't imagine why employers (or most prospective students at most schools, for that matter) would care about the quantity and quality of knowledge production at a particular institution. I care, society ought to care, universities and law schools ought to care, but employers don't and shouldn't care much -- and I don't think anyone has seriously made a case otherwise.

Well, I've convinced myself of the best way to do this, and it wasn't a close call -- evaluate the quality of the education "program," and use that for 100% of the score I give each school. How exactly I do that is a separate matter, but I'll save that for another day. Scholarship won't count at all.

But one thing does give me pause: I've just gone through this quick and dirty exercise of arguably Dworkinian interpretation -- trying to decide what to do, how to decide, by figuring out my role and trying to bring some coherence to the social practice of voting for the US news rankings. But maybe this was a silly exercise. Maybe the easiest thing to do, is just to look to the leading voice in the legal academy and the media on how to make the US News rankings better.

Maybe I should just ask myself: what would Leiter do? Professor Leiter, if you're out there and have a chance, I'd love to hear from you and others (including students and lawyers who hire law school graduates!) -- and even if not, I'll try to answer the question tomorrow in my next post.