« Law in Popular Culture | Main | Is Blogging the New Resume? »

Sunday, August 04, 2019

More on Malcolm Gladwell

Mike Dorf critiques the fourth season of Malcolm Gladwell's Revisionist History, which contains several episodes relevant to law and legal education. In particular, the first two episodes criticize the LSATs as the gateway into legal education. The basic argument is that the test's tight time constraints favor "hares" who think and react quickly over "tortoises" who take longer to think and analyze a problem, while the practice of law is more for tortoises.

But at least the first two episodes (I am midway through Ep. 3) are worse than Dorf suggests. The problem is that Gladwell tends to pick a thesis, find evidence that undermines one variable in furtherance of that thesis, then conclude (or assume) that his thesis has been proven, without exploring the other variables or other obvious explanations for the result. Dorf describes this as Gladwell "overclaim[ing]." Three glaring examples in these episodes.

The first is the story of Sixth Circuit Judge Jeffrey Sutton. Sutton attended The Ohio State University for law school because he did not get into Michigan, because it is assumed (without stating) his LSAT scores were not high enough. From this, Gladwell assumes that Sutton is a tortoise. And Sutton might have been overlooked for a clerkship by Justices such as Scalia (to whom Sutton was assigned as a clerk for the retired Powell) who hire based on law school (which is based on hare-favoring LSAT), which rewards hares over tortoises. This despite the fact that Scalia praises the tortoise Sutton as one of his best clerks because clerking and judging are jobs that favor tortoises. So the LSAT--and thus law schools, judges, and law firms--fails to identify, benefit, and reward brilliant tortoises such as Sutton.

There are many gaps in this logical progression. First, poor LSAT performance does not necessarily mean Sutton is a tortoise; it may mean he is a hare who had a bad day (the episode does not say how many times Sutton took the test). Second, we do not know that his LSAT scores kept him out of Michigan, as opposed to his undergrad grades or quality of his essay (Gladwell relies on the strong inference from LSAT being such a significant part of the admission calculus). Third, Sutton has some hare-like qualities--he did extraordinarily well in law school, which (Gladwell argues in Ep. 1) is framed to favor hares by using time in-class exams (this was even more true when Sutton was in law school in the late-'80s, where it was one end-of-semester in-class exam).*

[*] The reliance on timed exams in the first year is why the LSAT predicts 1L performance--both rely on time constraints in testing.

The second is a story told by Bill Henderson (Indiana-Maurer-Bloomington) about a 3L who booked his class, which relied on a take-home exam (I think it was 8 hours); this was that student's highest grade in law. The horror, Gladwell says. The school had labeled this person as an average student  by giving hare-like exams, when he was just a tortoise who, given the chance to take his time, could write a beautiful, well-organized essay.

Again, however, note the gaps and assumptions. I agree with the basic idea that ideal grading combines times and untimed assessments and I try to give both kinds within any class. But I have not not noticed a massive divergence in performance on the two types--good students do within a small range of well on both, weaker students do within a range of less well on both. And the testing format does not necessarily alone explain this one performance. Maybe the student related to that course's material (Gladwell does not identify the course, although Henderson teaches corporations and bus-org classes). Maybe the student responded well to Henderson's teaching style and learned well from him, which might produce stronger exam performance. We do not know how other IU professors assess--whether Henderson was the only prof giving take-home exams--and we do not know how this student performed in any other classes that used something other than in-class exams.

The third story discusses law practice. Gladwell describes a study showing that success in big-firm practice (especially rainmaking) is not correlated to the law school attended--the most successful attorneys attended night law school and schools the person never heard of. But big law firms continue to hire based on law-school prestige to bring in people who "look like them" (what Gladwell's subject called "mirrorocracy"). But that assumes that the people doing the hiring at big firms are the t-14-grad partners rather than the most successful attorneys at those firms--the hiring partners are t-14 grads looking for more t-14 grads, while ignoring the success of their partners who are non-t-14 grads but do the best work. But why aren't those non-t-14-grads who are the best lawyers also doing the hiring? Gladwell does not say.

Finally, not a criticism but a question: Gladwell describes the high enrollments of the three most prestigious schools in Canada--Toronto has 70k undergrads, British Columbia has 50K+, and McGill (the small, intimate school) has 25k. How do they avoid the problems that plague similarly large U.S. pubic schools--not enough teachers, over-reliance on adjuncts, massive classes, limited writing assignments that do not prepare students for the next level, etc.?

At some level, criticizing a podcast for not following and tying down every argument is similar to doing that to a blog post or twitter thread (although not fully--the first two episodes ran 79 minutes, time enough to tie-up obvious loose ends). But Gladwell purports to uncover the real story and offer real understandings, theories, and solutions to problems. It seems fair to hold him to the obvious flaws or incompletions in his arguments.

Posted by Howard Wasserman on August 4, 2019 at 01:48 PM in Howard Wasserman, Teaching Law | Permalink

Comments

I am actually sympathetic to Gladwell's basic point, as shown when I wrote "I agree with the basic idea . . ." My criticism is of Gladwell announcing Q.E.D. based on the examples of Sutton and one student in one class at IU.

As for reliance on one end-of-semester exam: It is changing, although not as rapidly as I believed. ABA accreditation standards want schools to do "formative assessments" in at least some courses. About 1/2 the 1L faculty at my school do some sort of in-semester evaluation. Something similar should be going on at other schools, although a discussion group at SEALS suggested that many of these interim assessments are not graded. So while many classes still place all the weight on one in-class exam, it is at least somewhat different, and still changing, compared with 1987-90.

Posted by: Howard Wasserman | Aug 7, 2019 8:22:28 AM

I would add, for what it is worth, that at Cornell I *always* beat the grade curve when there was a paper-writing component to the class, regardless of the subject-matter or the professor. But I rarely beat it (and often did not equal it) when the grade was based solely on one three-or four-hour final examination. So, I am sorry, but I am somewhat more sympathetic to Gladwell's point than you appear to be.

Posted by: Rob T. | Aug 7, 2019 7:38:18 AM

"(this was even more true when Sutton was in law school in the late-'80s, where it was one end-of-semester in-class exam).*"

Excuse my naïveté, but has this practice really changed that much since then? And especially in first-year law classes?

Posted by: Rob T. | Aug 7, 2019 7:28:47 AM

Rainmakers at successful law firms are generally not looking for rainmaking skills when they recruit from law schools, for the same reason that college coaches recruiting high school athletes are not looking for coaching skills.

Posted by: arthur | Aug 5, 2019 6:07:11 PM

Gladwell is infuriating. His MO, as far as I can tell, is:

Step 1: take interesting idea that has had currency in the academic world for decades,
Step 2: simplify idea for consumption by general audience, but don't acknowledge that idea is basically, old news among those who have paid any attention to the general field
Step 3: present idea as an incredible insight that the world has been ignoring, and most importantly, DO NOT GET INTO THE WEEDS. Alternative ideas or theories? Those are weeds. And so are caveats and qualifiers. The weeds must be avoided because it is important to the Gladwell brand that the listener/reader come away saying "Wow, I thought Problem X was really complicated and nearly impossible to solve, but now I know that, the solution is not only easy, but has been under our noses all along!!"

His three latest podcasts on casuistry range from prosaic to simply foolish. The last one in particular on police shootings has little, if anything to do with analogical reasoning. At bottom, he's simply saying, "when we decide how to address a police shooting, we should pay attention to the facts." For some real academic work on casuistry as a form of reasoning: see Sunstein's work on incompletely theorized agreements: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2544359

Posted by: Jack Preis | Aug 5, 2019 1:13:47 PM

"Gladwell describes a study showing that success in big-firm practice (especially rainmaking) is not correlated to the law school attended"

Why would we expect there would be a correlation? Big firms are not hiring a random selection of students from schools. Instead, the criteria are school specific. A big law firm may, for example, hire people from the top 2/3 of the class at Harvard but only the number one person from U of Illinois. It is therefore unsurprising that this stellar U of Illinois candidate performs as well as the average hire from Harvard.

Posted by: Kevin A Smith | Aug 5, 2019 5:06:20 AM

Not odd to you that Scalia said he would never have hired Sutton directly? And that he told the AU students that he would never hire any of them? I took that as a hint that the underlying theme is that the law is managed by a series of more or less arbitrary sorting mechanisms. Why exactly do you give your student tests? So they can be ranked.

Posted by: J. Bogart | Aug 5, 2019 1:11:44 AM

Well said: "The problem is that Gladwell tends to pick a thesis, find evidence that undermines one variable in furtherance of that thesis, then conclude (or assume) that his thesis has been proven, without exploring the other variables or other obvious explanations for the result." Shamefully, his books are the same. But he (or his publisher) understand most of America's attention span. As do politicians. For them, one is enough and any one will do,

Posted by: DJ | Aug 4, 2019 8:34:06 PM

Whatever problems with Gladwell, the larger problem of the LSAT is real. This is a test that really doe determine a large portion of your career path. Many who go to law school do not understand this. There is an incentive for law faculty to reinforce the importance of this test because it is such a large factor into their admission to Harvard, Yale, etc.

Posted by: anon | Aug 4, 2019 7:24:44 PM

Well said.

Posted by: Howard Wasserman | Aug 4, 2019 4:46:31 PM

I had some similar thoughts here:
https://reason.com/2019/07/10/malcom-gladwell-took-the-lsat-but-what-did-we-learn/

Posted by: Orin Kerr | Aug 4, 2019 2:40:30 PM

The comments to this entry are closed.