SCOTUS and Science

I would like to start by thanking Dan for inviting me back to Prawfsblog. I found the discussions and conversations that took place last time I was posting here very, very helpful, and I am really looking forward to them again.

I’d also like to thank Dan for letting me depart from the usual guest-posting format: rather than posting for a month, I’ll be posting sporadically throughout the year, on a schedule not set by me but rather by the Supreme Court. In my more self-congratulatory moments, I think of my goal for this year as ambitious; in my more honest moments, nuts. It is simply this: I want to read every Supreme Court opinion issued on cases heard this term and next (Fall 2011 and Spring 2012), and to evaluate each opinion’s use of empirical evidence.

At one level, this must sound crazy, not to mention somewhat arrogant. I’m an economist, and one whose expertise (such as it is) is confined to a particularly narrow area of criminology. The Court, meanwhile, hears cases that touch on nearly every branch of the social, biological, and physical sciences. Who am I, then, to critique their evaluation of, say, environmental science or forensics? A completely valid question, and I guess my answer ultimately comes down to this:  My goals here are fairly modest. First, I want to highlight each instance where the Court makes an unsubstantiated empirical claim, evaluate whether it seems easy to find reliable, rigorous evidence about the particular proposition being made, and to consider the claim’s importance to the case’s outcome. Doing so shouldn’t require that much field-specific knowledge on my part. Second, when the Court does bring real empirical evidence to bear, I do want to consider whether it seems reliable or not, and here the risk of overreach on my part is perhaps a bit stronger. But statistical techniques are not that different across fields, and the Court likely isn’t going to be relying on profoundly complex, cutting-edge empirical studies. Plus, the good thing about doing this publicly on a blog is that when I get it wrong I can expect commenters to point that out to me.

So what do I hope to accomplish with all this? At least three things:

1. I think it will be helpful simply to document the Court’s use and misuse of empirical evidence. I am certainly not the first person to be concerned about how the Court (and other courts) uses empirical evidence. But most of the work has focused on the Court’s misuse or non-use of empirical evidence in select high-profile cases. One of my goals this year is to examine just how pervasive a problem this is. Does the Court frequently make empirical statements without any validation, or which are actually incorrect? How important are these statements: are they generally used as additional support for a more-doctrinal claim, or are they outcome-determinative? Evaluating the opinions publicly on a blog will be incredibly helpful—I look forward to commenters criticizing my criticisms of the Court’s use of scientific evidence, especially those with statistical training in areas other than mine.

2. This project will also help me think more deeply about how non-scientific legal actors can best use scientific evidence. This was the issue that was the focus of my last set of posts on Prawfs, and it is one that concerns me more and more each day. The Justices and their clerks are remarkably well-educated and intelligent people, and by governmental standards have relatively light workloads (and thus have substantially more time to try to find the best scientific results out there). Yet it is likely* that they will often misuse empirical evidence, or rely on dubious assumptions about how the world works rather than on hard evidence. If these actors struggle with statistical results, then surely judges and clerks in lower federal and other state courts, not to mention all sorts of legislative and executive actors, must find that task even more challenging. What should empiricists do to make it easier for legal actors to use scientific evidence? Can we restructure institutions such as the Court to reflect the increasing importance of empirical evidence? After all, nothing in Article III says that Supreme Court clerks can only be recently-minted lawyers. More controversially, why shouldn’t judges (and Justices) employ experts of their own more often (or, in the Court’s case, at all) to assist them with complex scientific and empirical evidence?

True, the Justices often get “Brandeis Briefs” that try to point to empirical support for one position or another, but there are two reasons to take only little comfort in these. First, I would expect that the Court often gets competing Brandeis Briefs, again forcing the Justices to undertake a task for which they are poorly trained, namely separating out good empirical work from bad. And second, to the extent that such briefs are effective, they are substantially less prevalent in lower and state courts, where almost all cases are resolved (the Supreme Court hears about 80 cases per year, while in 2008 106 million cases were filed in state trial courts and over 300,000 in state appellate courts, along with nearly 300,000 in federal district courts and over 60,000 in federal appellate courts in 2009). So these are a limited solution at best.

3. I want to think more about how should we handle the “known unknown.” We should expect to encounter situations in which there simply isn’t any reliable empirical evidence for a particular question, yet the Court has to reach a decision—today. What should it do? Should it make an empirical guess? Can it do so without creating problematic “judicial facts” that take on precedential weight? And is there any way for the Court, if making such a guess, to do so in a way that makes transparent the error cost analysis driving its choice? Perhaps when guessing is problematic the Court should try to rely on more doctrinal solutions, but what if it is the type of case that simply requires a policy judgment to resolve?

I think these are all incredibly interesting and important questions in the abstract. But I also think it will be substantially more productive to wrestle with them in concrete applications over the course of the Court’s term. And I think it will be great to do publicly here on Prawfsblog, since these issues implicate a wide range of scientific fields, not to mention numerous doctrinal issues in evidence, fed courts, civil procedure, and beyond—in short, in many areas about which I know quite little. So it will remarkably helpful for me to be able to draw on what the readers of this blog know.

I will kick all this off in my next post by looking first at a case from the recent past, the Court’s 2009 decision, Caperton v Massey. Looking at whether judges should be forced to recuse themselves based on campaign contributions, the case contains a long dissent by Roberts which lays out a powerful argument against asking Justices to rely on empirical evidence. But, tellingly, Roberts in turn justifies his anti-empirical approach with a host of implicit—and in at least one critical case, almost certainly incorrect—empirical assumptions of his own. Roberts’ stumble only highlights the claim made by Mirjan Damaska that the current scientization of the law is the greatest structural challenge the law has faced since it replaced trial by divine proof with the trial by reason (and jury).

 

* This will not be the last time I make the obvious I-know-I’m-being-hypocritical disclaimer, attacking the Court for making unsubstantiated claims with unsubstantiated empirical assertions of my own. Like I said, being rigorously evidence-based is hard. Really hard.