« Fred Schauer, the NYPD needs your help! | Main | An Occupy Wall Street Constitution? »

Thursday, November 17, 2011

Caperton v Massey II

In my previous post, I pointed out that Kennedy relied on numerous problematic bald-faced empirical assertions in his majority opinion in Caperton v Massey. I now want to turn my attention to Roberts’ dissent. Unlike Kennedy’s opinion, Roberts’ is much more sensitive to the challenges posed by empirical claims or, perhaps phrased more carefully, by judicial decision-rules built on empirically-verifiable standards. Yet while accurately laying out the difficulties such decision-rules create, Roberts’ dissent is ultimately unsatisfying. The dissent itself relies on numerous unfounded empirical claims, and its final conclusion could even be read as “the Court can’t really do the job it is supposed to do.”

Roberts criticizes Kennedy’s test by listing forty questions trial and appellate judges would need to ask to properly operationalize it. Many of these are legal and doctrinal (scope of discovery, standard of review, etc., etc.), but at least eleven focus on social science questions:

  1. How much money is too much?
  2. How does the source of the contribution (from an individual vs. bundled from many) matter?
  3. Does the amount at stake in the particular case matter?
  4. How long does the “probability of bias” last?
  5. How many issues are implicated? If “disproportionate” funds are used to elect a “tough on crime” judge, from what sort of criminal cases (if any) must the judge recuse himself?
  6. Do judges feel a “debt of hostility” toward opponents of their candidacies?
  7. Does personal friendship between a judge and lawyer give rise to a “probability of bias”?
  8. Should the rule vary from state to state depending on historical baselines of judicial campaign spending?
  9. To the extent causation matters, how do we evaluate the causal link between the donation and the vote?
  10. Does incumbency matter, since incumbents already have an electoral edge?
  11. If the donor supported a different candidate during the primary, does that dilute the “debt of gratitude”?

These are all very tough questions, they are all ultimately empirical (or at least require some data to answer), and in many cases I imagine the data simply aren’t there, at least without some hefty extrapolation to the case at hand.

At the end of his list of concerns, Roberts concludes:

Today’s opinion requires state and federal judges to simultaneously act as political scientists (why did candidate X win the election?), economists (was the financial support disproportionate?), and psychologists (is there likely to be a debt of gratitude?). … The need to consider these and countless other questions helps explain why the common law and this Court’s constitutional jurisprudence have never required disqualification on such vague grounds as “probability” or “appearance” of bias.

In this particular instance, Roberts’ argument may not be particularly problematic. Earlier in his dissent he cites a passage from Tumey v Ohio, in which the Court states:

All questions of judicial qualifications may not involve constitutional validity. Thus matters of kinship, personal bias, state policy, remoteness of interest, would seem generally to be matters merely of legislative discretion.

In other words, the cases that rise to due process violations—direct financial stake in the outcome, say, or presiding over a criminal contempt trial that arose because of the defendant’s earlier actions toward that very judge—seem to come close to the parachutes-save-lives type situations: cases where the empirical assumption, even if unverified, seems so likely to be true that we are okay with the assumption. For all the non-parachute cases, though, the Court can leave the fact-finding to the legislature, which is generally thought to be a better fact-finder than the Court. That seems at least facially reasonable (although I am sometimes skeptical about legislative competence as well).

But. There may arise cases in which the Court has to make the call without being able to defer the non-parachute cases to legislatures or agencies. And in those cases, the nature of Roberts’ argument becomes more problematic. He is basically saying: we are not competent to make political science, economic, and psychological calls, and thus we should not do so. Clearly the latter half of that sentence follows nicely from the first. If you can’t do something, you shouldn’t do it.*

But that first half troubles me. The fact of the matter is that the types of questions the Court faces often force the justices to take on the roles of political scientists, economists, and psychologists. And Roberts is right that Supreme Court consisting of nine justices with no advanced training or practice in any of those fields is not well-suited to tackle that task. But the solution isn’t for the Court to just refuse to do its job—in fact, as I’ll show in a moment, Roberts himself acts like a political scientist and psychologist in this very dissent.

No, the solution must be for the Court to confront its epistemic failings directly. And it strikes me that there are two general avenues to consider. First, the Court could figure out how to become competent in the fields it must be competent in, perhaps by hiring clerks with a wide range of academic training and professional experience in policy- and science-related fields,** or by figuring out a way to bring in outside expertise for guidance (akin to the special masters, independent experts, and technical advisors that federal judges can appoint to assist them with technical issues). Or second, we could significantly restrict the range of cases the Court can hear to just those that involve statutory interpretation, matters of legal doctrine, or other such issues that implicate almost no empirical fact-finding. 

(That is, clearly, a beyond-cursory set of proposals. I will be fleshing them out in the posts ahead.)

But to allow the Court to admit it cannot handle social scientific evidence while still allowing it to do seems crazy. If my accountant walked in to a meeting and said “I don’t know how to add or subtract, but let me manage your finances” I would fire him on the spot. Yet at some level that is exactly what Roberts did.

Moreover, twice in the very same opinion, Roberts cannot adhere to the rule he appears to support, namely that the Court should be wary of confronting non-parachute empirical questions. First, he starts off his argument by stating that:

There is a “presumption of honesty and integrity in those serving as adjudicators.” All judges take an oath to uphold the Constitution and apply the law impartially, and we trust that they will live up to this promise. [Here he quotes an older Kennedy opinion, Republican Party of Minn v White: “We should not, even by inadvertence, impute to judges a lack of firmness, wisdom, or honor.”]

The first sentence is quote from Withrow v Larkin, and thus it could be seen as Roberts relying on precedent. But since Withrow doesn’t cite any empirical evidence to bolster its claim, I don’t see why it is entitled to any deference for its empirical statements. But the second sentence is Roberts writing for Roberts, and that sentence is an empirical claim as well. Why should we trust that judges will always be impartial? That’s a remarkably high bar for a human to clear—just consider the research about how people quickly become loyal to randomly-chosen teams or colors.*** Judges are no less human or mortal than the rest of us, and thus equally vulnerable to partiality. Of course, to complicate matters more, it is also possible that institutional pressures or senses of professional identity might help judges be more aware of the risks of partiality. The point is, Roberts’ point of departure is an empirical psychological claim, one baldly made with no empirical support, and one that quickly appears vulnerable.

Roberts then concludes with another unsubstantiated psychological/political science assertion. His concluding paragraph says:

It is an old cliché, but sometimes the cure is worse than the disease. … I believe that opening the door to recusal claims under the Due Process Clause, for an amorphous “probability of bias,” will itself bring our judicial system into undeserved disrepute, and diminish the confidence of the American people in the fairness and integrity of their courts.

Roberts recently criticized legal scholarship for being relatively useless to judges, but it seems like he needs to read a bit more by Tom Tyler. The work by Tom and other social psychologists on “decision acceptance” suggests that parties are more willing to accept decisions that go against their self-interest the more they think the procedures that led to them are fair. Perhaps allowing those concerned about bias to voice their complaints might strengthen their faith in the system, even if they lose, and denying them that hearing could make them more cynical. I don’t know for sure. But I do know two things: (1) Roberts' claim about bias challenges and faith was the very sort of psychological or political science claim he seems to argue the Court should avoid making (just look at the list of criticisms he made about Kennedy’s opinion and note how many apply here); and (2) his claim was empirically naïve.

If you’re still with me, thanks. To just summarize: Roberts’ concerns about the difficult types of questions the Court faces are valid. His fear that the Court is not well equipped to handle these social science questions is completely right. But as his own opinion makes clear, the justices simply cannot avoid these types of issues, no matter how hard they try. So the solution must be to find a better way to do so, not to simply admit incompetence.


* This is a major problem that empirical legal studies faces. Programs like Stata make it far too easy for those with little or no statistical training to run regressions. The resulting papers are often deeply flawed and thus not only fail to advance empirical knowledge, but by muddying up the evidence base actually retard it. That law review editors lack the competency of true peer reviewers to filter out the junk from the non-junk only exacerbates this problem. But that is a complaint for another (and earlier) day.

** I have always found it somewhat incredible that the default for Supreme Court clerks is someone with one year of post-JD experience, and with that experience being clerking for a federal appellate judge. Given the potential importance of Supreme Court opinions, I would like to think that the justices would surround themselves with far more skilled, seasoned assistants.

*** In a post complaining about bald claims, I know I need a cite for this. I have someone looking for it--I know I read this "somewhere"--and I will update this post with the link once I have it.

Posted by John Pfaff on November 17, 2011 at 12:06 PM | Permalink


TrackBack URL for this entry:

Listed below are links to weblogs that reference Caperton v Massey II:


Re note ***, you are probably looking for the minimal group paradigm studies conducted under social identity theory. See e.g., Tajfel, H., & Turner, J. C. (1979). An integrative theory of intergroup conflict. In W. G. Austin & S. Worchel (Eds.), The social psychology of intergroup relations (pp. 33-47). Monterey, CA: Brooks/Cole; Billig, M., & Tajfel, H. (1973). Social categorization and similarity in intergroup behavior. European Journal of Social Psychology, 3, 27-51; Tajfel, H., Billig, M., Bundy, R. P., & Flarnent, C. (1971). Social categorization and intergroup behaviour. European Journal of Social Psychology, 1, 149-178.

Posted by: Erik | Nov 18, 2011 12:43:46 AM

I like Professor Tyler's work on procedural justice, but I wonder whether it will apply that well to the kind of "hot button" Supreme Court cases that tend to make the news (like the upcoming health care decision).

A relatively recent line of research on the moral mandate effect, suggests that moral outrage at a decision can overwhelm the fair process effect that Professor Tyler has demonstrated.

See Elizabeth Mullen & Linda J. Skitka, Exploring the Psychological Underpinnings of the Moral Mandate Effect: Motivated Reasoning, Group Differentiation, or Anger?, 90 J. Pers. & Soc. Psychol. 629 (2006); Linda J. Skitka, Do the Means Always Justify the Ends, or Do the Ends Sometimes Justify the Means? A Value Protection Model of Justice Reasoning, 28 PERS. & SOC. PYSCHOL. BULL. 588 (2002); Linda J. Skitka, Christopher W. Bauman, & Elizabeth Mullen, Morality and Justice: An Expanded Theoretical Perspective and Empirical Review, in 25 ADVANCES IN GROUP PROCESSES 21 (K.A. Hedgvedt & J.Clay-Warner eds. 2008) (citing Mullen & Skitka (2006)).

Posted by: Stuart Ford | Nov 17, 2011 1:50:08 PM

There is plenty of support for the general position that people are largely incapable of being unbiased and rational. See Ziva Kunda, The Case for Motivated Reasoning, 108 Psychological Bulletin 480, 493 (1990); Eldar Shafir and Robyn A. LeBoeuf, Rationality, 53 Annual Review of Psychology 491 (2002) (arguing that the assumption of human rationality is misguided and presenting summaries of dozens of experiments that show that humans have been found to act in ways that are not rational); Shailendra Pratap Jain and Durairaj Maheswaran, Motivated Reasoning: A Depth-of-Processing Perspective, 26 Journal of Consumer Research 358, 358 (2000) (arguing that the motivation to arrive at a preferred conclusion enhances the use of those beliefs and strategies that are most likely to yield the desired conclusion); etc., etc.

There is also some support for the proposition that this applies to judges (judges are people too!). See Eileen Braman and Thomas E. Nelson, Mechanism of Motivated Reasoning? Analogical Perception in Discrimination Disputes, 51 American Journal of Political Science 940, 940-941 (2007) (noting that judges believe that the law guides their decision making, despite evidence that political preferences provide a better explanation for judicial outcomes than the law).

P.S. Yes. I cut and pasted these cites out of an article I am working on. :)

Posted by: Stuart Ford | Nov 17, 2011 1:44:41 PM

The comments to this entry are closed.