« LSAC Report on Best Practices | Main | Amanda Frost on Chief Justice Moore and the "Inferior" Federal Courts »

Thursday, February 12, 2015

Thinking Further About Cognitive Effort: Some Additional Thoughts on the "Simms Postulate"

My previous post explored the connection between the “closeness” of a legal issue and the level of cognitive effort that goes into its resolution.  In particular, I introduced an idea called the “Simms Postulate.”  Named in honor of a dubious but thought-provoking assertion that Phil Simms once made about the NFL’s “indisputable video evidence” rule, the Simms Postulate posits a positive correlation between cognitive effort and the closeness of an issue (or “issue-closeness” for short), holding that the harder a decision-maker works to resolve an issue, the more plausible it becomes to characterize the issue as “close,” “disputable,” “on the borderline,” etc.  The goal of the post (football pun intended) was to suggest that the Simms Postulate might be and indeed has been used when judges conduct doctrinal inquiries that turn on the closeness of an issue that has already been decided on its merits. 

I have thus far reserved judgment both as to the validity of the Simms Postulate itself and as to its utility as a tool of legal analysis.  But let’s now open that door.  Specifically, this post identifies and discusses five questions that strike me as potentially relevant to the overall value of the Simms Postulate. To those of you expecting a comprehensive and definitive normative conclusion, I must apologize in advance:  What follows is tentative and conjectural, aimed more at beginning an evaluation of the subject rather than completing it.  To those of you who like to read short blog posts, I should also apologize. I really didn't intend for this one to go on for so long, but, alas, it may now be eligible for the so-called “tl;dr” treatment. With those caveats offered, however, let me share some highly preliminary thoughts:

(1)  Does cognitive effort always signify issue-closeness?

The answer to this question has to be “no.”  Just because an individual has labored over the answer to a legal question does not mean that reasonable minds may disagree as to what that answer should be.  For one thing, high cognitive effort may simply signal a decision-maker’s unfamiliarity with (or inability to grasp) the law/facts that are implicated by the question itself.  Under those circumstances, high levels of cognitive effort may be expended, but only for the purpose of realizing that the answer to a question turns out to be fairly straightforward.

Somewhat more interestingly, even “expert” decision-makers with firm knowledge of a subject might sometimes end up devoting significant cognitive energy to resolving an issue whose answer turns out to be clear.  The truth of Fermat’s Last Theorem is now beyond doubt, but it took mathematicians over 350 years to show why.  I suppose that’s another way of saying that complexity is not the same thing as closeness: Some problems might be very difficult to solve ab initio, but once the solution emerges, no other answer is possible. Now whether there exist distinctly legal problems of this sort strikes me as an interesting question, but to the extent that such problems exist (perhaps, e.g., certain calculations of tax liability under the Internal Revenue Code?), then the complexity/closeness distinction is worth bearing in mind.

Still, even if cognitive effort does not always signify closeness, it might still prove to be a good enough indicator of closeness, at least in some circumstances.  So, the absence of an ironclad link between the two variables doesn't necessarily disqualify the Simms Postulate across the board.

(2)  Do we need a proxy for issue-closeness?

Phrased less kindly, this question asks whether the Simms Postulate poses a solution in search of a problem.  If it turns out that the relevant decision-makers are fully capable of asking directly whether a given constitutional right is “clearly established,” whether a given legal claim is “frivolous” or “substantial,” whether an agency’s reading of a statute is “reasonable,” etc., then why bother using an indirect proxy instead?  Even if valid, the Simms Postulate may not be needed; at best, it would simply complicate a set of inquiries that judges are already well-suited to perform.

The answer to this question depends in part on the findings of human psychology, a subject that falls outside the scope of my limited expertise.  Theoretically, though, the findings would have to show (or do in fact show?) that direct estimations of issue-closeness are likely to be biased or distorted in a systematic way. (Perhaps, for instance, I am hardwired to resist the sort of cognitive dissonance that would arise from suggesting that an issue I myself have decided in one way might reasonably have gone the other way.)  And if the findings did not indicate any such bias, then any need to rely on the Simms Postulate would indeed become less pressing.

What I might propose, however, is another way of framing the question that doesn't stack the deck so heavily against the Simms Postulate. Rather than ask whether it should displace a direct inquiry into “issue closeness,” we might more modestly ask whether the Simms Postulate could usefully inform such an inquiry.  When directly evaluating the closeness of a legal issue, a decision-maker will often look to several different variables: the language of the applicable text, the instructiveness of the applicable case law, how other judges have evaluated the closeness of analogous issues, etc.  Why not throw the added variable of “cognitive effort” into the mix? And indeed, if one revisits the examples I highlighted in my previous post, one sees the Simms Postulate functioning in this way, with the cited indicia of cognitive effort sometimes acting in concert with—rather than instead of—other variables that support the ultimate conclusion. If that is the relevant use of the postulate, then the urgency of the psychological question goes down. The usefulness of the Simms Postulate would no longer depend on a showing that judges suffer from systematic biases of other cognitive deficiencies when attempting to measure issue-closeness directly.

(3)  Do superior, alternative proxies exist?

Related to Question (2), we might wonder whether there exist easier or more reliable ways of approximating issue-closeness.  One immediate such candidate is the extent of disagreement that exists across a group of decision-makers.  Consider, for instance, the recent suggestion of Eric Posner and Adrian Vermeule—thanks to an earlier commenter for the pointer!—that judges might consider the votes and positions of their peers when evaluating the reasonableness of an agency interpretation. (Consider also the somewhat related suggestion of Vermeule and Jake Gersen that Chevron deference might be better implemented by way of a supermajority voting rule on multi-member courts.)  If, in short, “close” legal questions are questions on which reasonable minds might disagree, then the extentof judicial agreement or disagreement on the merits could in theory provide valuable information as to the closeness of the question itself.

Even if imperfect (and Posner and Vermeule do highlight potential complications with their approach), the “judicial-disagreement” metric may well be superior to the “cognitive effort” metric—superior enough, in fact, as to render the latter of limited usefulness.  On the other hand, the Simms Postulate might still remain useful in scenarios where only a single decision-maker has rendered a determination on the merits and thus lacks information as to other decision-makers’ views.  Furthermore, investigations into cognitive effort and investigations into judicial disagreement might sometimes operate alongside one another in a mutually supportive way.  Posner and Vermeule suggest, for instance, that one judge might sometimes wish to compare her own level of confidence about the rightness or wrongness of a position with the confidence levels of her colleagues, so as to gauge the depthof judicial disagreement in addition to its breadth.  And in that scenario, it still might be helpful for Judge 1 to ask whether Judge 2 struggled mightily with the issue or instead resolved it with ease.

(4)  How do you measure “cognitive effort”?

Three possibilities come immediately to mind. First, we might look to opinion length. Second, we might look to deliberation time. And third, we might look to first-person testimony. A few quick notes on each:

  • Opinion Length: The intuition here is that a lengthier opinion reflects a greater amount of cognitive effort than does a shorter opinion.  Notice that the claim is not that lengthier opinions require more effort to write—a point that is likely true but also immaterial to the question we are considering here. Rather, the claim is that we can infer from a lengthy opinion that the opinion-writer worked hard in deliberating over the outcome.  That may be true to some extent, but other variables might still complicate the inference. Perhaps the opinion-writer is longwinded.  Perhaps the opinion-writer wanted to opine on some matter of tangential relevance.   Or, perhaps the case simply involved a large number of issues, each one of which required little-to-no effort to resolve.  Interestingly—and on that last point—I’ve come across a few unpublished district court opinions that went of their way to attribute their length to the number of issues raised in a habeas petition; thus preemptively rebutting any Simms-inspired claim that the length of the opinion says something about the “substantiality” of the petitioner’s grounds for relief.  See, e.g., Peterson v. Greene, 2008 WL 2464273 (S.D.N.Y. June 18, 2008) (“The length of this opinion is a function of the number of arguments made by Peterson, rather than of the merit, or even difficulty, of any of them. None of the grounds he presents in seeking habeas corpus with respect to either of his convictions has the slightest merit. Accordingly, the petitions are denied. Because petitioner has not made a substantial showing of the denial of a constitutional right, a certificate of appealability will not issue . . . .”)
  • Deliberation Time: The intuition here is similar: when a decision-maker waits before rendering a decision, we might attribute the delay to an internal cognitive struggle: all else equal, the harder it is to decide a question, the longer one will wait before doing so.  Here too, however, delay may be attributable to any number of other factors: perhaps the decision-maker was busy working on other cases, perhaps the decision-maker was procrastinating, perhaps the decision-maker was agonizing over the stylistic aspects of an opinion, and so forth.  And the variable of deliberation time seems especially tricky as applied to multi-member bodies such as juries: True, a delayed verdict might indicate that all twelve jurors struggled with the question of whether to convict; but it also might indicate that a single stubborn juror held things up for a while.
  • First-Person Testimony:  Many doctrinal frameworks require judges to gauge the overall closeness of an issue that they themselves have already decided. So, if the evaluator of issue-closeness turns out to be the same person as the first-order decider of the issue, then that person might simple report on his or her own experience in deciding the issue as a means of justifying a subsequent decision regarding its closeness. “Trust me,” the judge might say, “I lost plenty of sleep trying to answer that question on the merits. Therefore, I conclude that the underlying claim was not frivolous.”  This metric carries the virtue of directness; but it is also susceptible to manipulation: The first-person decision-maker has privileged access to the workings of her own mind, and so is well positioned to exaggerate or downplay the degree of cognitive effort expended on the question.

A final point regarding all of these metrics: Recall my earlier observation that cognitive effort does not itself always signal issue-closeness.  So, even if long opinions, delayed judgments, or subjective descriptions tell us something about the level of cognitive effort that a judge has devoted to a legal problem, it does not necessarily follow that those problems qualify as “close” (as opposed to, say, “complex”).  That point, along with the difficulties I have attributed to each individual metric, suggests that exclusive reliance on the Simms Postulate is a risky business indeed. Rather, the postulate likely works better when accompanied by other independent measures of issue-closeness and/or substantive arguments concerning the nature of the issue itself.

(5)  Are there other factors to consider?

Of course there are other factors to consider!  For example, would an open embrace of the Simms Postulate induce first-order decision-makers to engage in unwanted strategic behavior? (E.g., “Because I want to deny qualified immunity, I’ll write a really short opinion on the merits and then point to that opinion to support my conclusion that the government official violated a clearly established right.”) Or might it simply confuse first-order decision-makers who otherwise might be trying to behave sincerely? (E.g., “Gosh, I’m taking a while to write this opinion. Does that mean the issue is more difficult than I initially thought? I guess I need to consider the issue further…”).  Should judges be more inclined to use the Simms Postulate when evaluating the closeness of an issue that they themselves have decided, or when evaluating the closeness of an issue that someone else has decided?  Are there other metrics of issue-closeness beyond the three I considered above? To the extent I do want to invoke the Simms Postulate, precisely whose cognitive efforts should figure into the mix? (e.g., When I am reviewing an agency’s interpretation of a statute, should I consider the amount of effort that agency officials expended on the interpretive question, in addition to the amount of effort that I myself expended?). And to what extent does the applicability/usefulness of the Simms Postulate vary according to the different ways that doctrines formulate and accord significance to issue closeness? (e.g., Does it make more sense to consider cognitive effort when considering whether a claim is "frivolous" than it does when considering whether an agency position is "reasonable," or when considering a government official has violated "clearly established law"?) And so on...

***

There is, I admit, something silly about all of this.  Real-world invocations of the Simms Postulate are infrequent at best and not likely to increase in frequency any time soon.  And, as my analysis suggests, this may well be so for good reason.  Perhaps the game simply isn't worth the candle, especially given that courts can and do make arguments about issue-closeness without in any way relying on the variables I have discussed in this post. At the same time, I figure that the Simms Postulate is in the air enough to justify some focused thinking about its underlying merits. Tackling the issue won’t win us any games, but it may at least allow us to move the ball forward and score a few analytical points.

Posted by Michael Coenen on February 12, 2015 at 11:35 AM in Judicial Process | Permalink

Comments

Let's take a concrete example, King v. Burwell (the healthcare tax credit case). The D.C. Circuit, in the panel opinion in Halbig v. Burwell, took 42 wide-margined but closely-typed pages (linked below) to ultimately conclude four words in ACA, "established by the State," were unambiguous. Excluding their discussion of the facts, standing, and cause of action, plus a summary of the parties' arguments, there are really just 14 pages on semantic unambiguity (at pages 16-29 of the opinion), divided more or less evenly between an interpretation of the provision itself, and interpretations of other provisions with similar or identical language where the government argued the challengers' interpretation of "established by the State" would lead to absurdities, plus 12 pages on whether legislative history and purpose created ambiguity (at pages 30-41). That's still quite a lot of work to find unambiguity in a short phrase, though note here that deciding which pages to count is itself pretty subjective.

Now, going through some of your factors, do we need a proxy for issue-closeness? Is there reason to think that cognitive biases are likely to distort judges' views on the ambiguity of a statute in the context of a highly politicized case (in either direction)? I'd say the answer to that is obviously yes - probably provably yes.

Are there superior proxies? Maybe not. Chevron as a voting rule might be superior, but it's not realistic, whereas it's conceivable that the Court could start using cognitive effort as a proxy for ambiguity. The suggestion that a diversity of interpretations is a proxy for ambiguity has some force, but one could argue that the effort it takes to argue that a statute is unambiguous one way is more suggestive than the existence of people who disagree, especially if the strongest claim they're willing to make is that the statute is ambiguous, rather than unambiguous their way. I very much doubt that the fact that some judges would find ambiguity is a great proxy for ambiguity. Differing claims about unambiguity, maybe. But it's always possible that one of those claims is badly misconceived, or relies on methodologies that the other side rejects. The Simms approach, on the other hand, looks at a claim of unambiguity from a perspective internal to that claim and helps us see whether, even under the methodology and premises of the people making the claim, it takes a lot of work to defend it. That might be better in some cases than the Posner/Vermeule idea.

What about the measure of cognitive effort? Can we really make much of the 26 pages it took the panel to find unambiguity; was the panel just shooting down the weak arguments for ambiguity before it? Here, cognitive biases are perhaps just as apt to color one's application of the Simms rule as to color one's reading of the statute. Someone inclined to find unambiguity would surely say that shooting down a lot of weak arguments is exactly what explains the lengthy opinion. Someone else might say that, e.g., the several pages (see pages 26-28 of the opinion) it took the court to show that the challengers' interpretation wouldn't make a mess of the qualified-individual provision because it turns out one doesn't really have to be a "qualified individual" to qualify to participate in an exchange are hardly, even on those pages' own terms, a take-down of a silly argument. Or that responding to the claim that the challengers' interpretation would create death spirals with a lengthy exposition of the similar death spirals Congress created in Guam (see pages 35-38 of the opinion) isn't quite the sort of thing one would say if one were brushing aside a weak argument. But at that point, I'm afraid, the proxy isn't doing any work anymore; one's simply assessing the strength of the arguments for ambiguity. Maybe the proxy is just a useful reminder that if you're fighting off a lot of arguments for ambiguity and going to some real trouble to do so, you might be looking at an ambiguous statute.

As for collateral consequences, I would seriously fear that if the Court made too much of the D.C. Circuit's 24 pages, circuits would try to cert-proof unambiguity findings by writing less and obscuring whatever arguments for ambiguity there may be. Or, having been given an incentive to write less, they'd write less and be less likely to discover the flaws in their reasoning by writing more.

On balance, then, at least in this case, I think the Simms idea has some real appeal but fairly limited utility, and perhaps even some real disutility.

http://www.cadc.uscourts.gov/internet/opinions.nsf/10125254d91f8bac85257d1d004e6176/$file/14-5018-1503850.pdf

Posted by: Anon | Feb 12, 2015 4:53:16 PM

Post a comment