Friday, July 11, 2008
The Legacies of U.S. News's Robert Morse, AALS's Susan Prager, and Us
Can we use the annual U.S. News survey of law professors to create a race to the top in legal education? In prior posts, I've explained why we should do it -- now I want to talk about who can do it and how.
At the 2007 AALS workshop on rankings, one of the breakout sessions was entitled: "if you can't join 'em, beat 'em," referring to the creation of rival rankings systems to U.S. News. But not only can we join U.S. News, we already have. U.S. News sends surveys each year to four professors at each of the 184 ABA-approved schools (deans, associate dean for academic affairs, chair of hiring committee, most recently tenured professor). Last year's response rate was 70%; by my fuzzy math, that's more than 500 law professors a year filling out these surveys. How are they doing it? By what criteria?
We don't know, but we do know that in the aggregate, all we are doing is spitting back the previous year's rankings. We all know it's time for a change, and here's who can help make the race to the top in legal education happen.
U.S. News's Robert Morse: To the methodology czar, all you have to do is add one word -- "educational" -- to clarify that when you are asking law professors (and practitioners, for that matter) to assess the quality of each school's program on a scale of 1 to 5, that you are referring to the educational program. Please also remind us that you changed the call of the survey question; I think you used to ask about a school's "reputation," but no more. Now you actually want us to assess the quality of each school's "program." We could use your help in creating this race to the top on education that your ranking system makes possible.
AALS's Susan Westerberg Prager: A few months ago, former UCLA Law Dean Susan Westerberg Prager was named the new executive director of the American Association of Law Schools (AALS). You are no doubt in the process of charting a course for your tenure there, and you can use this as an opportunity to define your legacy. As I understand it, the AALS's basic position since U.S. News started has been to fight to destroy the rankings. That may well have been the best strategy in 1990; it's not now. Rankings are here to stay, and U.S. News has no challenger in sight.
You can help take back the rankings by identifying an institution or firm that might be able to provide some useful information to survey respondents who are far too busy to fill out the survey responsibly, but are trying to do the right thing -- think of what we need as a "Voters' Guide to the US News Survey," or as Nancy Rapoport put it, a Law Schools Rated Almanac.
To start, here's a modest suggestion: Dean Prager, you can appoint a new AALS committee on rankings, which might have as part of its charge coming up with "best practices" for law professors filling out the annual U.S. News survey. If I were on the AALS rankings committee, I'd be inclined to recommend a few "best practices":
(1) Rate schools based on the degree to which it "adds value to the personal and professional lives of its current and former students." Much of this, then, should be an assessment of relative educational quality, but it also should include some sense of help in getting jobs, and the strength of the alumni network.
(2) Rate schools relative to one another in the relevant market in which they compete for students, and in which employers choose whom to hire. It does no good to compare Harvard to Baylor, or give Harvard, Yale and Stanford all a "5." You must differentiate among schools in the relevant market.
(3) When you are filling out the survey, remember your audience -- students trying to choose among schools to attend, and employers choosing from which schools to hire -- and try to give them some useful information. All things equal, Baylor or Texas Tech? Stanford or Yale? William and Mary or Washington and Lee?
Law Professors: We complain about U.S. News, but we essentially control the rankings -- 25% is our survey, the biggest piece by far. Right now, we're throwing a good opportunity away. We talk about alternative rankings, and multiple rankings are good. But we have a ranking system that everyone uses, and we're already participating. Let's take advantage.
I don't mean to blame past U.S. News voters -- no one has the time to realistically assess the quality of each school. It's a collective action problem, Prisoner's Dilemma, really-serious-game of-Twister issue, whatever. The point is: some institution or group of people needs to do a "Voters' Guide to the U.S. News Survey: Adding Value for Students" so that busy law professors and deans who want to fill out the survey in a responsible way can do so.
By failing to do anything with our 25% of the rankings except replicate the previous year's rankings, we are ensuring the competition takes place on buying LSAT scores and other stupid law-school tricks. Without real competition based on quality, changes like those of counting part-time students' LSAT scores, or the ABA changing how it asks for job placement numbers, are magnified. With a real competition in quality, such changes become much less important.
We've got some work to do, and the next survey is just a few months away. Will you help? (firstname.lastname@example.org) I'll commit a bit of time on this if others are interested, but I can't and won't do it alone. People can help in different ways -- to make this happen, for example, it will help to have support from prominent scholars and former deans who say this is a good idea. I'm guessing there are only so many crazy junior law professors at non-elite (though terrific) schools willing to get involved, and it's not clear that anyone will follow.
More next week about how one might construct a "Voters' Guide to the U.S. News Survey" based on a relative, within-market assessment of each school's capacity to add value for students. Have a good weekend.
TrackBack URL for this entry:
Listed below are links to weblogs that reference The Legacies of U.S. News's Robert Morse, AALS's Susan Prager, and Us:
How about some pressure on career services offices to inject a little honesty into the numbers they report? Students would face honor code (or equivalent) violations if they behaved with the same indifference to truth that most law school career services offices display.
Posted by: Mopping floors at BigLaw? Counts. | Jul 11, 2008 10:57:56 AM
Agreed. One virtue of a "Voters' Guide" of the kind I'm envisioning, that resides on the Web, is the opportunity for students to have input into assessing the quality of a school, and to help truth-squad the data their school provides to the ABA and U.S. News. Now, students at a particular school will of course have countervailing pressures: "Do I tell the truth about how terrible this school is, or do I say how wonderful to help our ranking and the value of my degree?"
But students really ought to have the most interest in this kind of competition on "value-added," greater accountability, etc. And it won't happen unless students help push for it and make it happen.
Posted by: Jason Solomon | Jul 11, 2008 11:48:40 AM
Interesting post, but one of the problems is that so few professors vote. The only time a professor who isn't a dean votes is upon receiving tenure, and if multiple people at a school get tenure that year, only one of them gets to vote -- or so I have been led to believe.
So a voter's guide would be great, but who's really voting? US News ain't a democracy, and it does not get a good representative sample of voters. I never got a vote for US News, and unless I become a dean, I'll never ever get a vote. DC residents have more voting power than most professors in the US News rankings.
So can we really take back the rankings? Maybe if we actually had a vote, but most professors don't. . .
Posted by: Daniel J. Solove | Jul 11, 2008 7:58:18 PM
>Rate schools based on the degree to which it "adds value to the personal and professional lives of its current and former students."
How are law school professors suited to rate how various law schools across the country add value? Wouldn't that require not only knowledge about the career paths of students at all sorts of schools, but their capabilities before law school as well? This would be an absurd measure of correlation as causation. Success after law school: do you attribute it to the student or the school? Students from school x are on average more successful. Does school x add more value or just get better students?
Posted by: will | Jul 11, 2008 9:36:47 PM
Dan, good point, but you do have a voice, and a blog, and colleagues with votes (dean, assoc. dean for academic affairs, hiring chair, most recently tenured), and I hope you'll use all of the above to both promote this way of approaching the U.S. News survey, and offer help on how to actually do it. Thanks.
Posted by: Jason Solomon | Jul 11, 2008 11:40:38 PM
It is interesting how poorly the alternative rankings have fared so far.
Posted by: Stephen M (Ethesis) | Jul 12, 2008 8:49:36 PM
We can agree that the purpose of the USNews report is to create a chart that simultaneously tells employers who to give the best jobs to and studenst where to go to get the best jobs, right? then why, other than a desire to feel like you're an important part of the lsat to work cog, would you want the survey to be about anything other than that? Rather than parse about for hours trying to get the phrasing just right, make the question real simple: Thinking solely of his/her future career successes and ignoring personal geographic preference, where would you want your child to go? The reason that alternative rankings fail is that none of them does a demonstrably better job of answering this question to make up for the increased informational costs in finding them. most of them, I suspect, are worse.
At the same time, there's certainly is a world of improvement still to bring about in terms of making clear certain issues of rankings clustering and regional strength, to say nothing about the previously mentioned truth in advertising problems. Obviously, the regional kind of improvement is the sort that is going to require real work on the part of whoever does it, which means it's probly not going to happen. If anything, this is the biggest flaw with usnews because its cookie-cutter approach treats all law schools as though they were part of some sort of national legal market that doesn't exist outside of the elite. Any visitor to law school bulletin boards is certainly aware of the tragic sorts of questions which plague the rankings-obsessed student such as "I got into Wisconsin and Baylor, but I want to stay and work in Texas. How much will the rankings drop from Wisc hurt me?" i suspect that any adequate attempt to reconcile this would involve a complete redesign, which doesn't seem forthcoming, rather than an amateur effort by professors to rate schools according to how they compare to their close substitutes in the marketplace. What would that mean anyway? Would you rate Yale a 5 and GULC a 2 while giving St. Louis University a 4? How totally broken would that be?
Posted by: C | Jul 13, 2008 1:37:08 PM
I agree with Mopping Floors at Biglaw counts. Schools do not accurately report post graduation employment, nro do career services offices service students.
I attended a top 50 school and have yet to find permanent employment over a year since graduating in 2007.
Moreover, students and recent grads should be the ones ranking the schools, they are the ones who ahve paid for the educationa dn shouldered a debt burden only to be thrown to the wolves in terms of employment.
Posted by: Angry Grad | Jul 14, 2008 1:56:03 PM