Systematic Reviews, Conflict Resolution, and Party Control

In a recent post, I argued that systematic reviews are perfectly compatible with at least two goals of our adversarial system: finding the truth and avoiding arbitrary outcomes. In fact, I believe that they advance these goals better than adversarial procedures (putting aside for now--I will address this in my next post--the critically essentially issue of where the review's guidelines come from in the first place). In this post, I want to examine two goals of adversarialism that are in tension with systematic reviews: conflict resolution and party control

That systematic reviews do not necessarily facilitate conflict resolution and party control is easy to see. Reviews can frequently take as long as one or two thousand hours to produce, or about 150 to 300 eight-hour workdays; independent experts will likely not be able to dedicate half a year full-time to such a task, to the total number of days will run even longer. And by definition the independent expert producing the report is outside the control of the parties--and, in fact, is more reliable than the partisan expert precisely because he is outside the control of the parties.

There is no real way to reconcile systematic reviews with these goals, so here I want to take a completely different tack: when it come to cases involving complex scientific evidence, we may want to prioritize accuracy and non-arbitrariness over conflict resolution and party control.

To see why, I want to reject a common defense of party control. Cecil and Willging quote a federal judge as saying the following, when explaining why he or she dislikes independent experts: “We’re conditioned to respect the adversary process. If a lawyer fails to explain the basis for a case, that’s his problem." It is a common sentiment, but despite whatever merit it may have had in the past, it is completely wrong today.

Ours is a lightly regulated economy; even the almost-certain upcoming wave of reforms are unlikely to change that, especially outside of the financial sector. We call on our court system to act as both regulator and social insurer. Such a role means that a lawyer failing to explain his case hurts more than just his client, and a bad jury verdict can resonate far beyond the particular plaintiff and defendant.

Tort law provides a clear example. The National Vaccine Injury Compensation Program was created at least in part from a fear that systematically excessive jury verdicts would eliminate vaccines altogether (excessive in the law and economics sense: vaccines provide a net social surplus even when the harms are taken into account, but the damages exceed the harms by enough to eat too deeply into the legitimate profits to make production worthwhile). A few bad cases could have had industry-wide ramifications.

(As a side note, I am afraid to think of what would have happened had the recent panic over vaccines and autism--a panic that may be helping measles spread in England, where it really caught on--resulted in litigation before a jury rather than before the Program's special masters, who dismissed the link as wholly without scientific merit.)

There is a general informational externality in cases. When the courts get the science wrong in one case, all firms in all industries (though perhaps more strongly the more closely linked the industries, or type of evidence, are) adjust their beliefs about the likelihood of a scientifically accurate verdict in cases they may face in the future. If juries too readily accept arguments of causation, those who produce relatively non-harmful products will be over-deterred; if juries are too quick to dismiss legitimate causation, those who produce relatively harmful products will be under-deterred. Either way, the implications of one case can reach far beyond it, and such externalities need to be acknowledged alongside speedy resolution and party control.

And this is not an issue unique to torts. In light of the National Academy of Science's blistering critique of forensic evidence, we should expect lawyers to attack everything from eye-witness testimony to fingerprints to truly unreliable techniques like "forensic odontology" (bitemark analysis). In the absence of some sort of officially regulatory body to assess forensic techniques--and the Innocence Project is actually pushing for just such a body--it will fall to the courts to regulate forensic science. Assuming that outcomes are path dependent, the quality of the scientific decisions reached in early cases could have huge implications on subsequent outcomes. There is something to be said for dragging out these early cases, for putting the social good ahead of the individual.

The effects extend beyond the injurers to the victims as well. Court cases, at least high-profile ones, can act as important channels of risk communication. If the courts are reaching bad outcomes--were they to sanction the link between autism and vaccines, say--they are communicating risk badly. And at the very least, where the courts and the scientific community in general (over-anthropomorphizing slightly) are in conflict, that conflict only adds to the swirl of competing claims, which make it hard for people to know what to believe.

And note that restricting the type of scientific evidence available does not undermine other, more consequentialist goals of party control. Stephan Landsman, for example, argues that party control allows more innovating claims to reach the courts. Parties remain free to think up any sort of cause of action they desire. They simply can't rely on incorrect or insufficiently reliable evidence to support it. Landsman also points out the importance of voice: that simply being allowed to speak has great value, a point supported by the evidence in Tyler and Huo. But again, we put limits on what exactly people can say all the time: the rules of evidence are all about balancing probative and prejudicial. 

Plus, under current practices, systematic reviews would not impede voice at all. If produced by special masters, they would be used to assist the judge in his Daubert proceedings, but they actual presentation to the factfinder would be done by the partisan experts who survive review. If produced by a Rule 706 expert, they would be presented alongside--not in place of--partisan expert testimony.

Nonetheless, we don't want to ignore the importance of speed and control. Some cases don't involve the type of evidence that demands a systematic review--we don't need to synthesize high-quality RCTs to determine whether wearing a parachute reduces the risk of death when skydiving--and in these cases we should focus more on conflict resolution and party control. And I am more confidence that this is the type of screening function that judges may be more competent to undertake. It may be hard in some borderline cases for judges to decide whether the evidence is sufficiently compelling to mitigate the need for systematic reviews, but that error rate is likely lower than that when judges try to assess the quality of evidence directly.