« Introduction to Judicial Bullshit | Main | Open Letter from Jewish Law Professors Protesting the Treatment of Professor Katherine Franke »

Thursday, May 17, 2018

The sensational hype over lawless law school admissions

There's been a lot of hype about the proposal to end of the requirement that law schools use the LSAT in admissions. Some sources (here unlinked) fret about standardless admissions in law schools and a race to the bottom.

There are many reasons to doubt this. But I wanted to take a few (?) paragraphs to look at the recent past of the LSAT and the transition we may be experiencing.

Current admissions standards require admission of applicants "who appear capable of satisfactorily completing its program of legal education and being admitted to the bar" (Standard 501(b)), which "may" include considering admissions test scores (Interpretation 501-2). Beyond that is a separate requirement for a "valid and reliable admission test" (Standard 503). If a school uses something other than the LSAT, the school "shall demonstrate that such other test is a valid and reliable test" (Interpretation 503-1).

My eyes sometimes glaze over reading string cites to ABA standards, so I'll break in here. These rules simply put schools in the position of having to demonstrate that it's okay to use another test, like the GRE. Inertia, then, has been a major reason why schools mostly stuck with the LSAT. They always could've done something else, but it would require some work.

When the University of Arizona took the lead considering the GRE a few years ago, it had to carry a new burden to prove compliance. But given the sharp decline in applicants and LSAT test-takers over the last decade (improving this year), it seemed like a sensible move. Here was a pool of test-takers more than twice as large as the LSAT pool. Capturing a small percentage of them would be a tremendous advantage. Other schools soon followed suit.

But the burden to "demonstrate" that the test is "valid and reliable" has caused some concern. To start, the ABA hasn't offered much in the way of explaining what the evidence would look like. Schools could offer their best evidence from students who took both the LSAT & GRE, but others were worried whether that might be enough. ETS, which administers the GRE, came out with its own study to show that the test was as reliable, but then others wondered whether that would be enough.

It's worth pausing to note that there are already exemptions from the LSAT requirement. While schools under Standard 503 must require each applicant take a test, there's no rule as to how much weight that test score should receive (Interpretation 503-2). If you are a law school at a university admitting a student from your undergraduate program or another graduate program, you can admit that student without an LSAT score as long as the student scored in the 85th percentile of the ACT/SAT/GRE/GMAT, or was in the top 10% of the class, or had a 3.5 undergraduate GPA (Interpretation 503-3).

Furthermore, the LSAT is less effective than it once was. LSAC reports the highest score, which is less reliable than the average score of repeaters (UPDATE: it's worth emphasizing that schools also still receive the average of all scores and each score from each test; the ABA and USNWR permit using the highest score instead of the average, too); it now also permits unlimited retakes. LSAT is still a much better tool than, say, undergraduate GPA, and it offers (some) pretty good predictive value.

But the recommendation approved this week from the Council that may become final soon would abolish Standard 503 and all its interpretations. Instead, there'd just be 501: demonstrate that you have sound practices. It would also add an interpretation, newly renumbered at Interpretation 501-3: "Failure to include a valid and reliable admission test as a part of the admissions process creates a rebuttable presumption that a law school is not in compliance with Standard 501."

It's not terribly different from the previous rule, except that instead of "shall demonstrate," it's simply that schools have a presumption to rebut--no need to demonstrate a valid and reliable admission test. Demonstrating compliance by showing, say, low attrition rates and high bar passage rates may well be enough. Or, showing that your alternative test is valid and reliable would mean there's no presumption to rebut. (It's worth noting this also abolishes those other rules about taking from your own undergraduate institution--it's all in the same holistic bucket.)

I doubt this will suddenly be a race to the bottom--the bar pass rates and recent enforcement from the ABA should prevent much of that. So, the experimentation can truly begin. That I emphatically support. But will they?

I think many schools will be reluctant to do too much too drastically (but, I think, a few brave ones will!). Inertia matters. Rebutting a presumption matters. And risk-averse schools may become nervous about big changes to their admissions programs, only to see attrition one year later or bar passage rates three years later worsen and having to undo policies.

But beyond all this, USNWR will likely (unfortunately) drive a lot of decision-making. USNWR already converts GRE scores into their percentile equivalents for reporting LSAT medians. I'm not familiar with any evidence that this practice is warranted--I don't know if a 75th percentile score on the GRE is the same, worse, or better, than the 75th percentile on the LSAT. But, to the extent schools are driven by USWNR, they are likely to keep GRE scores, on percentile terms, roughly correlated with LSAT.

It also means that how USNWR in the future decides to handle bodies of students without any test score will be the true test of experimentation. Of course, if brave schools choose to do what they think is best for their students, USNWR be damned... but that's the stuff of rampant speculation.

Posted by Derek Muller on May 17, 2018 at 11:40 AM in Life of Law Schools | Permalink

Comments

Thanks for this, Derek! FWIW, we (Dave Killoran of Powerscore and I) had a very similar take on how it may evolve and the USNWR driving forces.

https://blog.spiveyconsulting.com/aba-on-track-to-drop-lsat-requirement-what-happens-now/

Posted by: Mike Spivey | May 17, 2018 11:52:16 AM

Derek, dozens of schools have made SAT/ACT scores optional for undergraduate admissions. One could argue (as they do) that they are doing. as you put it, for what they think is best for their students. More disinterested (and cynical?) observers suggest that it instead is a way to increase the applicant pool without lowering their competitive ranking by USNWR. Would be applicants presumably take the usual standardized tests, then forward them to the college or not depending on whether it will help or hurt their case.

Why would it be any different for law schools?

Posted by: PaulB | May 17, 2018 12:29:08 PM

Since what is taught in law school has little relevance to the bar exam, and indeed most students take a separate cram course specifically for the bar, perhaps we should consider having students take the bar before law school. Then there’d be no chance of wasting three years and hundreds of thousands of dollars on a credential that couldn’t be used.

Posted by: Brad | May 17, 2018 6:44:23 PM

Paul,

I teach at one of those test-option undergraduate schools, and while I don't know for certain which students opted out of the SAT/ACT, I think I can guess. Many students come in writing essays that show a lack of basic familiarity with the grammar rules the SAT and ACT test. Perhaps they just forgot them, or they're simply not putting as much care into their work, but from our in-class discussions I suspect they never learned them at all.

We're now facing a situation where there's rampant high school grade inflation and outright grade fraud, combined with a lack of standardized testing. It's nearly open enrollment. And of course undergrad institutions have their own grade-inflation issues (pro tip: minor in a subject that gets very few students; you'll be given a good grade to ensure you take future classes and prop up their enrollment numbers). At least going to an "LSAT or GRE" system keeps some standardized testing involved, but I wouldn't be surprised to see testing eventually removed as a requirement for law school.

Posted by: Derek Tokaz | May 22, 2018 10:41:28 AM

Post a comment