« Arts as Game | Main | Anyone Starting with Heller? »

Tuesday, August 26, 2008

U.S. News and Part-Time Program "Gaming"

The front page of today's WSJ has an article about a possible change in the U.S. News law school ranking formula.  The change, which has already been much discussed, might require law schools to count the LSAT scores and GPA of part-time students in schools' overall LSAT and GPA numbers. 

I'm sure others will have more to say about this than me.  I just want to draw your attention to one curious sentence in the article: "Mr. Morse of U.S. News says the magazine will run tests of how the change would play out in rankings, and then decide in January."  Now, the sentence does not precisely say that the U.S. News formula decision will depend on "how the change would play out in rankings," but it sure seems to imply it.  Otherwise, why are they running the tests?

Presumably, U.S. News wants to be sure that such a change in methodology would not make the rankings seem less legitimate.  But if there is some gold standard that U.S. News uses to see how well its tests come out, then just show us the gold standard law school rankings!  Perhaps there is some other explanation.  Perhaps one can test the "stability" of the rankings or some other set of criteria that are neutral with respect to the rankings of particular schools.  But, it will not be shocking to suggest that the primary impetus for the testing may have more to do with the business goals of U.S. News rather than the development of an unbiased, careful approach to measuring law school quality.

Posted by Adam Kolber on August 26, 2008 at 08:49 AM | Permalink


TrackBack URL for this entry:

Listed below are links to weblogs that reference U.S. News and Part-Time Program "Gaming":


The adoption law is so wonderful for the general people in this country.This australian writing service lets you learn how to write paper description for explanation your ideas on adoption and child care laws.The general people should abide all these laws.

Posted by: Daniellesherry | Oct 30, 2019 8:57:29 AM

Today’s post deals mostly with process: starting at the beginning, but with the end in mind. What is the goal for the final moot court brief? A polished, professional, persuasive product that fully addresses the issues presented in the problem. As I mentioned in the Question Presented post, find the relevant competition’s score sheet if available, and use it as a guide. Reviewing past winning briefs is also very helpful. It pays off to spend some time gathering resources.

Posted by: Daniellesherry | Oct 30, 2019 8:53:39 AM

Necessary reading: "The Empirics and Ethics of USNWR Gaming" by Henderson and Lipshaw over at the Legal Profession Blog http://lawprofessors.typepad.com/legal_profession/2008/08/posted-by-jeff.html#comments

Posted by: Patrick S. O'Donnell | Aug 27, 2008 9:34:42 AM

Wha? Purity? S.c.cutus, please tell me you're joking and not some kind of plainclothes facist with a license to practice.

Posted by: Kevin | Aug 27, 2008 12:26:36 AM

I'm sure they just want to see if this change will lead the rankings to change drastically. If it does, that casts serious doubt on the validity of the rankings to begin with. If this change leads to a number of small changes, it seems like "fine-tuning;" but if it knocks half the second tier into the third, or whatever, consumers should wonder why they are trusting these rankings at all.

Posted by: Heloise | Aug 26, 2008 9:06:39 PM

About time. Schools have been gaming the system like this for too long. They need to be punished for accepting people with sub-165 LSATs at all. A good law school is about intellectual purity.

Posted by: S.c.cutus | Aug 26, 2008 12:18:40 PM

It's possible to perform tests on a the new method of ranking without having a "gold standard" to test them against. There are at least two reasons.

First, it's possible to have a testing system that isn't itself generative. This is common with regression testing; U.S. News may be checking to make sure that the change doesn't fundamentally break their rankings -- e.g., we could probably all agree that the exact opposite of their 2008 ranking would be wrong.

Second, the tests could be exploratory. People who work with large datasets run new kinds of queries precisely because they don't know what will happen and they're curious. If we make this change, what happens? Hmm, why is it that we see a correlation between X and Y when we do? This calls for further investigation. In the process of seeing what happens when they change the formula in this way, they might discover other patterns that could call for other adjustments.

Posted by: James Grimmelmann | Aug 26, 2008 10:08:34 AM

The comments to this entry are closed.