« Who Cares Whether Cake-Baking Is "Expressive"? The Doctrinal Costs of Focusing on Private Burdens Rather Than Governmental Purpose | Main | Reply Brief on the Geography of Campaign Finance Law »
Monday, September 11, 2017
More on Corpus Linguistics and the Criminal Law
When people ask me why I became a law professor, I have an endless list of reasons at my disposal: Teaching is an outrageous amount of fun. I get paid to write and think about interesting ideas. No one ever expects me to wear a pants suit. But a major reason that I enjoy being a law professor is that I absolutely love to disagree with people.
So imagine my delight when I saw how many comments my original post on Corpus Linguistics and the Criminal Law received. Not only did many of the commenters engage with my essay, but the commenters included Steven Mouritsen (who wrote the first law review article suggesting that judges conduct corpus linguistics analyses and who has a forthcoming article with Justice Thomas Lee in the Yale Law Journal on the topic), Neal Goldfarb (who runs a blog on Law & Linguistics and who attended the 2107 BYU corpus linguistics symposium), and Brian Slocum (who also attended the BYU conference and whose co-authored paper formed the basis for my essay advocating against the use of corpus linguistics to interpret criminal laws )
The comments are all quite good, and many of them have helped me refine some of my concerns with corpus linguistics. But I also want to push back against others.
First, the comments appear to reveal a rather stark divide in the corpus linguistics community: While some are pushing corpus linguistics analysis as a new method of statutory interpretation, others are far more modest about what corpus linguistics can provide. In his comment, Brian Slocum repeated the modest approach that he and his co-author take in their paper. Neal Goldfarb (who was kind enough to submit multiple comments) said that “corpus linguistics can have a very significant impact on issues of statutory interpretation,” but he says that his use of corpus linguistics in not intended “to remake the theoretical framework of legal interpretation.”
Having read Brian Slocum’s most recent essay, I was already aware that not all scholars who conduct corpus analyses necessarily endorse it as a method of statutory interpretation. But the comments thread suggested that even corpus linguistics staunchest legal supporters may have a more nuanced view of how the tool is to be employed. In particular, Steven Mouritsen seemed uncomfortable with my characterization of corpus linguistics as a method of statutory interpretation that treats ordinary meaning as an empirical question and then answers that question with evidence of how frequently a term is used a particular way. I had read his previous work as suggesting that corpus linguistics was not merely an additional tool that judges might wish to employ (or not, whatever they might wish), but rather as a new method of interpretation—a variation on textualism. But as I’ve read and re-read his comments, I am actually not so sure that he is backing away from (what I read as) previous methodological arguments. In particular, his comment includes the following passage:
“I don’t think anyone is advocating (I certainly don’t advocate) merely characterizing the most frequent sense of a word as the ordinary meaning. That would be arbitrary. But corpus linguistics can allow you to examine the way in which a word is used in a given syntactic, semantic, and (sometimes) pragmatic context, in the speech or writing from a given speech community or linguistic register, and from a given time period. To the extent that you find that a given sense of a word is overwhelmingly more common in a particular context similar to that of the statute, in a relevant speech community or register, and from a similar timeframe, I don’t think it is an extraordinary leap to conclude that the people subject to that statute would have understood the word in a way that is consistent with its most common meaning in those circumstances. This is a presumption that I would think should be rebuttable where there is compelling evidence that an alternative sense of the word or phrase was intended. And I don’t advocate (and I don’t think anyone in the pro-corpus camp advocates) foreclosing consideration of other evidence of meaning simply because the corpus data suggests a particular answer.”
It is a little unclear to me whether Mouritsen is backing away from a statement that he and Lee made last month on the Volokh Conspiracy that “a complete theory of ordinary meaning requires us to take into account the comparative frequency of different senses of words” (emphasis mine). I don’t think that he is, because he is saying that the corpus analysis would create a “rebuttable presumption.” But nonetheless, it would be helpful if he (or perhaps he and Lee) could clarify whether they believe (a) that corpus linguistics searches and analyses ought to routinely be part of statutory interpretation or if their position is simply that corpus linguistics might sometimes be useful, and (b) whether, in addition to adding corpus linguistics data to the “plain meaning” step of textualism, whether they are advocating for any additional interpretive changes.
Second, I want to push back on the scientific/objective language that Mouritsen uses in his comments (and elsewhere) to advocate for corpus linguistics. He tells us that “one of the chief benefits of the corpus approach is transparency. When corpus linguists are wrong about ordinary meaning, they are transparently wrong, because their approach and their findings are replicable and falsifiable.” I see the rhetorical value of this language, but I have a very hard time understanding the analytical work that it does.
In the hard sciences, we place great stock in findings that are replicable and falsifiable. If, for example, someone conducts the same physics or chemistry experiment that I conducted, and their measurements are the same as mine, then we can be reasonably confident that my experiment was conducted appropriately and that its findings tell us something about the physical world. But that hard sciences model does not map onto corpus linguistics—at least not corpus linguistics as a method of statutory interpretation. That is because a corpus search is replicable, but the search itself doesn’t tell us anything about usage/meaning. It is the analysis that matters. To say that someone could conduct the same corpus search and obtain the same results is no different than saying someone could consult the same dictionary that I consult and find the same entries. But just as I might draw different conclusions from those dictionary entries, so too are people likely to draw different conclusions based on their corpus analyses.
Don’t believe me? Check out the Michigan Supreme Court’s opinions in People v. Harris, 885 N.W.2d 832 (Mich. 2016). The majority and the dissent come to the precisely opposite conclusions about the “ordinary meaning” of a statutory term based on the same corpus data. The corpus data does not (at least not as far as I can see) tell us which of the Justices in that case were “transparently wrong.” At the very least, the Michigan Supreme Court Justices didn't find the matter transparent.
Third, I am still left with my same concerns about notice and accountability that I raised in my original post (and which are fleshed out in more detail in the paper itself). Several commenters pushed back a bit on these concerns. And I’ll confess that I am quite certain that it is my role as a criminal law professor that makes these concerns so salient for me. Some commenters suggested that the corpus database is more likely to mimic public intuitions about meaning because the public reads the newspapers and magazines that make up the corpus. But judges read those very same sources. Why would the corpus database be an accurate reflection of a non-judge’s linguistic intuition, but not a judge’s intuition? I don’t disagree with those who say that corpus linguistics databases capture more information about language usage than does a judge’s own status as an English speaker. But whether the corpus more accurately captures the average person’s linguistic prototypes than does judicial interpretation is an empirical question about which I have seen no real data.
Mouritsen offers a few responses to my notice concerns. But I don’t find them very persuasive. He says he prefers the bean counting approach of corpus linguistics to judicial intuition because intuition has led to cases like Muscarello, which he finds arbitrary. I agree that Muscarello was wrongly decided. But we don’t need corpus data to know that the defendant’s interpretation of “carry” was entirely plausible (if not more plausible) and thus he should have benefited from the rule of lenity—Justice Scalia’s dissent did an excellent job of doing just this. Mourtisen acts as though corpus linguistics is a check on government power because it would have led to a different result in Muscarello, and he also points out that several other cases he has analyzed would have come out in the defendants’ favor. This approach misunderstands my critique. I made a structural argument about the separation of powers and how it requires a substantive role for judges in statutory interpretation. That structural argument derives its force and legitimacy from the Constitution, not whether more cases will be decided in favor of defendants. The argument is about constitutional design and it is about requiring judges to take responsibility for the substance of their decisions. I do not think it is a coincidence that Justice Breyer took refuge in a dictionary in Muscarello. He knew that the dissent’s interpretation of the term was more sensible and accepted; but by clothing his holding in “objective” analysis—like using dictionaries—he could avoid responsibility for the consequences of his decision.
Mouritsen does not deny that he is trying to check government power by decreasing the substantive role of judges to use their own judgment in interpreting statutes. But he is trying to avoid the notice and accountability concerns by claiming that corpus linguistics will lead to better outcomes for defendants. In particular, he claims that corpus linguistics will lead more judges to conclude that a term is ambiguous and thus requires the rule of lenity. Maybe that will happen over time. But I honestly doubt it. The Michigan Supreme Court split over whether a corpus search supported two inconsistent interpretations—neither side saw ambiguity. And I find Justice Lee’s decision in State v. Rasabout, 356 P.3d 1258 (Utah 2015), entirely too similar. He found twelve examples of one meaning of a term and one example of another meaning in a corpus analysis of 81 examples. So 68 out of 81 usages did not clearly fall within one meaning or another. Did he find that the term was ambiguous? No. He went on to further analyze what he admitted were unclear usages in order to try and fit them into one of the two contested meanings in the case. I find it difficult to read such a corpus analysis and think that Mouritsen is correct that corpus linguistics will result in more findings of ambiguity. (And, although I’m not a behavioral scientist, I wonder whether the representativeness heuristic tells us that corpus linguistics data is unlikely to lead judges to find ambiguity. I’d be interested to hear from those who are more familiar with behavioral economics—am I wrong to think that heuristic bears on this issue?)
Finally, let me emphasize that I understand why corpus linguistics is so attractive. Not only does it appeal to our current fascination with big data, but it also taps into dissatisfaction with the current textualist regime. My critique of corpus linguistics should not be read as a defense of the current approach to interpreting criminal statutes. I think that judges need to do a better job fulfilling their historical role as gatekeeper for punishment decisions. As I note in the essay, I advocate for a much more robust rule of lenity and more expanded and consistent clear statement rules. If these tools were expanded, they could bring about the predictability and objectivity that corpus linguistics appears to offer. And they would do so while promoting notice and accountability.
Posted by Carissa Byrne Hessick on September 11, 2017 at 01:01 PM in Carissa Byrne Hessick, Criminal Law | Permalink
Comments
My second post responding to Carissa is up at LAWnLinguistics: https://lawnlinguistics.com/2017/09/21/meaning-in-the-framework-of-corpus-linguistics/#more-1454
Posted by: Neal Goldfarb | Sep 21, 2017 5:45:47 PM
Professor Hessick,
Again, thank you for your thoughtful posts and to everyone who has commented above. This has been a valuable discussion, and I hope it will be helpful in, at the very list, identifying and clarifying points of disagreement. I disagree with your description of the approach that I (and my co-author) have described. I think corpus linguistics can be a very useful tool for providing information that can be relevant to some of the linguistically oriented questions that the law sometimes asks. I don’t think that it should be used to foreclose consideration of other evidence of meaning. While there may be methodological differences among the very few papers that apply a corpus approach, I disagree with the suggestion that what Justice Lee and I have advocated treats corpus linguistics as more than a tool that can provide useful information when the law asks certain linguistically oriented questions. In short, I don’t believe, and haven’t advocated, corpus linguistics as a “new twist on textualism.”
I wanted to, again, respond to a few of your comments. I won’t be able to respond to all of your comments, both here and in your essay, without violating the common law prohibition against improperly lengthy blog comments, but I would encourage any readers who are still following this post to read your excellent essay and also invite them to read my article with Justice Lee (currently a brisk 89-page beach read), and the contributions to the BYU symposium on Law and Corpus Linguistics, including yours, once they are published. Taken together, I think these will give readers a solid command of what law and corpus linguistics is about and also present a number of important criticisms of the approach.
(As before, my comments are my own and don’t necessarily represent the views of my esteemed, occasional co-author.)
You say: “It is a little unclear to me whether Mouritsen is backing away from a statement that he and Lee made last month on the Volokh Conspiracy that ‘a complete theory of ordinary meaning requires us to take into account the comparative frequency of different senses of words’ (emphasis mine).” I am not. Though that is not, in fact, what we said. We said: “[A] complete theory of ordinary meaning requires us to take into account the comparative frequency of different senses of words, the (syntactic, semantic and pragmatic) context of an utterance, its historical usage and the speech community in which it was uttered.” When you take into account each of these factors you may often find that a highly specialized, infrequent sense of a word is the most likely candidate for ordinary meaning. You may find that the context clearly eliminates some or all but one of the senses of the word as possibilities. You may also find that the word or phrase in question has many potential uses in that same context, in which case it is difficult to make the case that any of the potential senses is the ordinary meaning. So the characterization of the approach in your prior post that “if the database says a term is more often used as X than Y, then corpus linguistics tells us that is the ‘ordinary meaning,’” to me doesn’t capture the approach we have outlined in our paper. Yes, I think that the frequency with which a word gets used in a particular way, in a particular syntactic, semantic and (sometimes) pragmatic context that is similar to the legal text in question, in the speech community or linguistic register that is similar to the one of those subject to the statute, and from a similar time frame is a relevant consideration when determining the “ordinary meaning” of the words of a statute. As I noted before, “[t]o the extent that you find that a given sense of a word is overwhelmingly more common in a particular context similar to that of the statute, in a relevant speech community or register, and from a similar timeframe, I don’t think it is an extraordinary leap to conclude that the people subject to that statute would have understood the word in a way that is consistent with its most common meaning in those circumstances.” To the extent you have such evidence, I think it would be useful in resolving the question of ordinary meaning.
In a prior comment, I said: “I don’t think anyone is advocating (I certainly don’t advocate) merely characterizing the most frequent sense of a word as the ordinary meaning. That would be arbitrary.” Consistency may be the hobgoblin of little minds, but I did say the same thing in my 2010 student comment on this subject. And Justice Lee said the same thing in his concurring/dissenting opinion in In re Baby EZ—his first opinion relying on corpus data.
In my prior comment, I said: “And I don’t advocate (and I don’t think anyone in the pro-corpus camp advocates) foreclosing consideration of other evidence of meaning simply because the corpus data suggests a particular answer.” Justice Lee’s concurrence / dissent in the In re Baby EZ case is an example. The opinion concerned whether the Parental Kidnapping Prevention Act (the “PKPA”) applied to adoptions, or only to custody determinations pursuant to a divorce. In his opinion Justice Lee examines the statutory definition of ‘custody’ and ‘visitation,’ the text and grammatical context of the act, the Act’s constitutional implications (it is one of the few statutes enacted pursuant to the Full Faith & Credit Clause and the enabling act of that Clause), the statutory preamble and its stated purpose, the legislative history of the PKPA, and a clear statement rule about narrowly construing federal statutes that implicate traditional state law prerogatives like family law. These were all sources of relevant data about the meaning of ‘custody’ in the PKPA and appeared to point to the same conclusion.
I think it is fair to say that Professor Larry Solum’s recent paper “Triangulating Public Meaning: Corpus Linguistics, Immersion, and the Constitutional Record” is another example of this. Professor Solum suggests that taken together (1) corpus methods, (2) immersion in the linguistic and conceptual world of the drafters and readers, and (3) study of the constitutional record (its drafting process, the debates, and the early history of its implementation) are all points of data that can be “checked against the others” to attempt to “triangulate” original public meaning. Here again, corpus linguistics is presented as a source of relevant data about meaning, but is not intended to be the only source.
I think the dichotomy between, on the one hand, corpus linguistics as an interpretive methodology on par with textualism and purposivism (or even as you characterized it as “a new twist on textualism”), or, on the other hand, corpus linguistics as a source of “marginally” useful information, may not be an accurate way to frame the utility of linguistic corpora for interpretive problems. I think corpus linguistics can be a source of “very” useful information that can provide helpful insight in many cases in which the question that the law poses is a linguistically oriented question. Applications of the ordinary meaning canon are one example. And I don’t think that they are a necessarily “textualist” example. I can’t imagine that there is any jurisdiction in the United States that doesn’t use some variety of the plain or ordinary meaning canon of statutory interpretation, regardless of whether the judges who invoke it are textualists, purposivists, Dworkinian moralists, Sunsteinian minimalists, or fans of Nickelback. And I think that when judges pose the question of ordinary meaning, they are asking a linguistically oriented question (about how a word or phrase is used in a given syntactic, semantic, and/or pragmatic context, in the speech or writing of a given speech community or linguistic register and at a particular time). Obviously, that is not the way that judges frame the question of ordinary meaning, but you can often find elements of these factors referenced (and glossed over) in judicial opinions that invoke the canon. If these factors are relevant to the ordinary meaning inquiry (and I think that they often are), then corpus linguistics will often (though certainly not always) be able to provide useful information.
You disagree with the characterization that the corpus linguistics approach to ordinary meaning is scientific. You acknowledge that the search that gathers the language data is replicable, but argue that “it is the analysis that matters” and different parties can draw (and have drawn) different conclusions from the same data. I simply don’t agree that the idea that people drawing different conclusions from the same data, or subjectively interpreting objective data, undermines the scientific nature of a corpus-based the inquiry. If we are going to classify as “unscientific” any form of inquiry where two people interpret the same data differently (and even subjectively), then we would have to condemn economics and every social science as “unscientific.” We may, in fact, have to shut down many upper level, theoretical courses in the “hard sciences” where subjective interpretation of data may lead different people to different theoretical conclusions. Disagreement about the interpretation of data is part of what Thomas Kuhn called the “route to normal science.” As I noted in my prior comment, corpus data won't “tell you what to do with th[e] information or whether that information is helpful for resolving certain types of questions. Shared standards, practices, and methods emerge when people in the relevant field start using the tool and start debating where it is useful and where it is not useful (or even harmful).” The fact that two parties look at the same data and interpret the data differently doesn’t render the enterprise unscientific.
I think the Michigan Supreme Court’s opinion in People v. Harris is an example of this. In deciding whether the ordinary meaning of “information” was simply "true information" or included the notion of "false information," the majority cited corpus data arguing that “‘information’ is regularly used in conjunction with adjectives suggesting it may be both true and false.” The dissent argues from the same data that “the majority’s contention that the [Corpus of Contemporary American English] affords support for the proposition that the term ‘information’ is ‘regularly’ or ‘commonly’ modified by one of these adjectives. I find to the contrary. And where ‘information’ is unmodified by one of these adjectives, I believe it is overwhelmingly used to refer to truthful information. . . . I do not believe that a judicial interpretation of ‘information’ drawn from use of the term in ½ of 1% of all of its appearances in a corpus constitutes an ordinary, common, or reasonable interpretation of the term.” The argument appears to be about not what the data say, but about what “ordinary meaning” actually means (or “regular” or “common” meaning, which are both synonyms of “ordinary”).
This is what I mean about being “transparently wrong.” This is a poor choice of words and you are correct that it is not necessarily obvious from the data which of these opinions is right or wrong. My point is (as we argue in our paper) that the corpus data gives some content to these differing conceptions of ordinary meaning. It forces the judges to show their work and gives a transparent account of what each opinion appears to mean by “ordinary meaning.” This is in contrast to what happens when a court fails to explain what they mean by ordinary meaning and merely invokes the canon and it is not what happens when courts simply cite a dictionary (given that dictionaries don’t typically provide information about ordinary meaning and are typically cited, as James Brudney and Lawrence Baum pointed out in their article “Oasis or Mirage,” merely as window dressing).
Posted by: Stephen Mouritsen | Sep 20, 2017 5:13:27 PM
@anon: "I do not dispute that a substantial majority of the collocates of 'vehicle' have to do with ground things, including tanks, trucks, and cars. However, is that majority not simply explained by the fact that there are so many more trucks and cars than airplanes, or that there are so many more injuries by trucks and cars than by airplanes, that we simply talk more about trucks and cars instead of airplanes when we use the word 'vehicle'?"
I don't think the conclusion follows from the premise -- especially given that there is a separate word that denotes the category of planes, helicopters, etc.: "aircraft" which appears in COCA 14,790 times, compared to 41,096 for "vehicle." Although it's true that "vehicle" can be used to refer to things that fly, I suspect that it has that meaning only when there is something in the context that supports such a reading, as in the phrase "aerial vehicle."
If that's right, maybe "vehicle" has a default meaning whereby it denotes what are generally referred to as "motor vehicles" (cars, trucks, buses), but at the same time can be understood to refer to things that fly when that is required by the context.
Posted by: Neal Goldfarb | Sep 17, 2017 6:38:20 PM
Thank you for your response, Mr. Goldfarb.
I do not dispute that a substantial majority of the collocates of "vehicle" have to do with ground things, including tanks, trucks, and cars. However, is that majority not simply explained by the fact that there are so many more trucks and cars than airplanes, or that there are so many more injuries by trucks and cars than by airplanes, that we simply talk more about trucks and cars instead of airplanes when we use the word "vehicle"?
(Side note 1: Here is some data on the amount of goods transported by trucks, rail, and air. https://www.bts.gov/archive/data_and_statistics/by_subject/freight/freight_facts_2015/appendix_a/table2_1m. As you can expect, the data is completely lopsided towards trucks. Should we not expect at least a 7 to 1 ratio favoring trucks to airplanes, assuming, generously, that airplanes include all of "Air, air & truck," "Multiple modes," and "Other"? Of course, if you take airplanes to only include "Air, air & truck," then that would mean a 1,000 to 1 ratio of trucks to airplanes.)
(Side note 2: It does seem that "motor" is the highest collocate with "vehicle" across all times. However, I think the fact that the word "vehicle" is so often seen with "motor" does not mean that people think of cars and trucks when they think of "vehicle." Rather, I interpret it to mean that people regularly think of nonmotor vehicles as within the ambit of "vehicles" and that "vehicles" often need to be clarified with the adjective of "motor" when used to refer to cars and trucks.)
And I would not give so much trust to the COHA data, especially from the earlier decades of the last century. Again, referring to Lee/Mouritsen, they look to the COHA data from the 1910's. But, as you can see in their list of top collocates, numbers show up. Why? Because there are so few texts within COHA that even quoted numbers can become collocates. Indeed, of the 15 collocates between "motor" and "vehicles" from 1910's (and 15 is top of the collocate list!), 6 of them come from one single source, "The Army and the Motor Truck," written by one single author. There is so little within COHA that a single historical source with an outlier usage of a word can heavily skew the statistics.
I couldn't comment on the use of CL to deal with "personal" in FCC v. AT I have not done the necessary research on the provision. I do suspect that the word "personal" is less subject to the topicality problem than "vehicle," but I need to think a bit more about it. Certainly, with the Equifax breach being a top headline of this week and with the word "personal" being used in so many articles about the event to refer to individuals, there is some reason to think that the word "personal" may be subject to similar distortions.
Posted by: anon | Sep 15, 2017 3:20:06 PM
anon: While I'm not dismissing the issue you raise, I think that its significance in any given case will depend on what the issue is. For example, if the question is whether an airplane counts as a vehicle, the results that you give from the 1940s are worth considering despite being weighted toward the military domain, because the ones that are most relevant seem to all be associated with ground vehicles rather than aircraft.
Also, while looking at what collocates are the most frequent may be sufficient if they all point in the same direction (as in FCC v. AT&T, involving "personal"), in other cases it may be necessary to go into the data more deeply. For instance, looking at the top 200 collocates of "vehicle" in COCA (https://corpus.byu.edu/coca/?c=coca&q=58931966), we can see that the substantial majority of them look like they are associated with cars, trucks, etc. (though in some cases it would make sense to drill down into the underlying corpus lines to see what they show). There are a few that are associated with aircraft ("aerial") and spacecraft ("launch"). So the next step would be to compare (a) the number of collocate-types for the two categories, and (b) the cumulative number of corpus lines (collocate-tokens) in each category.
It looks to me like the results from COHA for the 1940s follow the same pattern (https://corpus.byu.edu/coha/?c=coha&q=58932219).
Posted by: Neal Goldfarb | Sep 15, 2017 1:54:54 PM
Hessick is absolutely correct that newsworthiness is a problem. To appreciate this, one only needs to conduct the Lee/Mouritsen search of "vehicle" in the 1940's. Here are the top hits:
1 MOTOR
2 ARMORED
3 TANKS
4 WHEELED
5 COMMERCIAL
6 REGISTRAR
7 WRECKED
8 HEAVY
9 ROAD
10 AMBULANCES
Is that a surprise, given what happened in the 1940's? Does that mean a statute in the 1940's with the word "vehicle" in it referred to tanks? No. It simply meant that people in the 1940's were talking a lot about war.
As I said before, normalization of the frequencies is important. And normalization, done properly, at least requires a deep understanding of the culture and of the events of the times. And this process of creating the normalization factors is so subjective that it will entirely destroy the supposed objectivity in using computational linguistics.
Posted by: anon | Sep 15, 2017 10:51:03 AM
A few quick thoughts in response to these comments.
I'm glad to see Daniel Ortner join the conversation--I enjoyed his article on CL and the Rule of Lenity. As for his response to the notice point, the argument I am making is not that members of the public have to conduct corpus searches, it is that those search results may diverge from their linguistic intuitions. As I explain in the paper, there is reason to think that the search results will be skewed by things such as newsworthiness.
As for whether CL will lead to more lenity, I understand the appeal of that argument. But if corpus data is designed to fit within the existing textualist scheme, then it will get priority over lenity and other canons of construction. And as for whether defendants will be able to use corpus data to convince judges that a term is ambiguous, our experiences in Rasabout and Harris may show that to be overly optimistic. Behavioral economists have shown that giving people a little bit of information may cause them to be (unwarrantedly) certain and to make worse decisions than with less information. In short, CL could make things worse.
I am having a hard time understanding the distinction that Asher Steinberg is trying to draw. He says:
"The claim isn't that an individual non-judge's intuitions are more likely to be consonant with the data from the corpora than a judge's; of course that's possible depending on what the judge reads and who he knows, but it's not the claim. Rather, the claim is that a huge sample of usage is a better measure of aggregate understandings than a few individuals' understandings, which are grounded in their much smaller half-remembered samplings of usage."
I’m not sure what it means to say that the corpus linguistics data is a better measure of “aggregate understandings.” Does that refer to the meaning most people are likely to understand a particular word to have? If so, then I disagree with his statement. The reason to distrust a person’s linguistic intuitions is not simply because he or she is only one person, but rather because of the systemic errors she is likely to make. (Neal Goldfarb’s new post at Law n Linguistics does a nice job illustrating this.) Those errors are not only shared by both judges and non-judges alike, but they are likely to make *most everyone* have a linguistic intuition that does not match up with the corpus data. Those systemic errors mean that the law of large numbers, Condorcet’s jury theorem, etc cannot save corpus linguistics from the notice critique. The problem is not simply judges vs non-judges. It is that the frequency with which a word is used a certain way is (at least in some cases) not going to match up with what more than half of Americans think the “ordinary meaning” of that word is. And when we are dealing with criminal laws, we simply can’t enforce laws if most people wouldn’t know what the statute means.
It is, of course, possible that Asher Steinberg means something else by his phrase “aggregate understandings.” If so, I’d be interested to hear more.
Posted by: CBHessick | Sep 14, 2017 6:14:58 PM
Tammy Gales and I have also written an article for the BYU Law Review issue on corpus linguistics and law. We regard corpus linguistics as a tool that can be sometimes helpful in determining ordinary meaning. In our article, we attempt to set forth conditions under which corpus analysis will be maximally efficacious.
(1) The court must determine that ordinary meaning should carry the day, rather than some less ordinary meaning. Consider Chisolm v. Roemer, a Voting Rights Act case. A majority of Supreme Court justices determined that even though the act bans racist voting schemes in the election of “representatives” and we do not typically speak of judges as “representatives,” it would be wrong to infer that Congress intended to create a safe harbor for racist voting schemes in the election of Louisiana’s supreme court justices. It thus construed “representatives” to include judges. It is not difficult to identify cases in which the justices decided that ordinary meaning should take a back seat to more specific information about what the statute was intended to accomplish.
(2) The court must determine what it means by “ordinary meaning.” Smith v. United States illustrates this problem. Smith attempted to trade his unloaded machine gun for illegal drugs. Was he “using” the weapon in the drug crime? A majority of six Supreme Court justices answered affirmatively. Justice Scalia dissented on the ground that when one thinks of using a gun one thinks of using it as a weapon. Both sides were right. Using a gun as merely a thing of value is an unusual way to use it, but once one does that, the word “use” seems to be a natural description of what happened.
(3) The court must perform the right search. Consider Muscarello v. United States. When Muscarello had a gun in one part of his car and drugs in another as he drove to a drug deal, was he “carrying a firearm” in the ordinary sense of that term? Justice Breyer conducted a search of “weapon,” “carry,” and “vehicle” in close proximity and found that when those words are used together they reflect carrying a weapon in a vehicle about one-third of the time, making that meaning ordinary. Stephen Mouritsen points out that the more meaningful search would have excluded the word “vehicle.” He conducted such a search and found that carrying a weapon on one’s person predominated.
(4) The court must decide what to make of the absence of a particular meaning from a corpus. Take Justice Brewer’s observation in Church of the Holy Trinity v. United States to the effect that we would rarely use the word “labor” to describe the activities of a member of the clergy, even though it is not wrong to do so. In some instances, including that one, the absence of a meaning from a corpus may reflect the fact that people would not be comfortable using a term in a particular way. In others, such as we just saw in Smith, the problem is less linguistic and more a reflection of the fact that some things happen more often than others, or at least we talk about some things more than others. This is a problem if the ordinary meaning rule attempts to capture the meanings with which the average speaker would feel comfortable using a term. Thus, when a meaning is absent from a corpus, we believe the best practice is to demonstrate that the missing meaning is reflected in language that does not incorporate the term in question if one wishes to refer to a corpus.
Taken together, these criteria provide a roadmap to best practices in using corpus linguistics in legal analysis.
Posted by: Larry Solan | Sep 14, 2017 10:05:02 AM
For those who are interested: I've posted the first of what will be two posts commenting on Carissa's paper.
https://lawnlinguistics.com/2017/09/13/some-comments-on-hessick-on-corpus-linguistics/#more-1399
Posted by: Neal Goldfarb | Sep 13, 2017 4:36:11 PM
Neal Goldfarb, you're right that you would need a corpus of texts from the relevant discipline. This is certainly not something we would want judges doing on their own accord even if they had the capacity to do so. But, technical term of art analysis is interesting because judges treat the question of whether a statutory term has a technical meaning as a question of fact. Parties---particularly from large industries where term of art analysis is most common--- would certainly have the resources to create a specialized corpus and through an expert witness, provide the court with concordance lines, collocates, etc.
While I do not think creating a corpus from scratch would be cost-prohibitive, linguists have created custom-made corpora for a number of industries thanks to the rise of "English for Specific Purposes" (ESP), a sub-specialty of ESL. Companies will create and use custom-corpora to discover what vocabulary/grammatical structures are most common in the field, and use the data to teach English to their foreign workers. Parties might be able to re-purpose such corpora for term-of-art analysis.
Posted by: James Heilpern | Sep 13, 2017 2:15:56 PM
On further thought (and speaking as someone with training in the physical sciences), the corpus linguistics in the Lee/Mouritsen article suffer from such poor characterization that it severely undermines their claim that judges and lawyers are capable of implementing the method on their own.
Let's focus on these two paragraphs:
"We can see from this data that the meaning of vehicle evolved significantly from this period, though the automotive use of vehicle still predominated. The decade of the 1950s is remarkable as the first decade in which the spacecraft sense of vehicle appears, but also the last decade in which the horse-drawn collocate of vehicles appears.191 Unmanned no longer appears, but manned vehicles appear (spacecraft in this case). Still, the overwhelmingly most common use of vehicle is the automotive sense, while a number of context specific possible senses are attested. And, again, none of the top fifty collocates of vehicle include the notion of airplane and bicycles."
"Our review of the corpus data also lead us to conclude that a number of other potential candidates for the ordinary meaning of vehicle are out of the running. Airplane (and its synonym plane) are unattested in the collocation data, and a review of the concordance data didn‘t turn up any instances in which the word vehicle was used to describe what we might think of as a conventional airplane. The corpus data did show a few well attested uses of vehicle in reference to unmanned aerial drones. This was not the most common or even a comparatively common use of vehicle, but the data demonstrated that it was an attested and possible use of vehicle. We can accordingly conclude that the ordinary meaning of vehicle is automobile—at least to the extent that we accept the most common use of a word and exclude other possible uses."
What's "overwhelming[]" about "automotive" in the first paragraph? Even using the data from the footnote, it shows a frequency of 58 for "motor" compared to 18 for "space." (Note that there are other colocates related to aerospace, such as "orbit" and "test".) Those numbers do not give me confidence to exclude "aerospace" from vehicles.
The authors also assert that "bicycles" do not show up as the Top 50. True, but "bicycles" show up at Top 84, with a frequency of 2, and the vast majority of the "Top 50" have frequency of less than or equal to 5. Certainly, "Top 43" through "Top 71" only have a frequency of 3, and the sentence, "none of the top fifty collocates of vehicle include the notion of airplane and bicycles," is highly misleading at best.
But more importantly, good characterization requires normalization. People mention things when they have a need to deal with or discuss those things. So, of course, "vehicle" refers most often to "automotive" stuff because people most often interact with "automotive" in their daily lives. An analysis based on the raw frequency count is therefore not interesting or illuminating until the frequency is normalized by some factor that accounts for how important that word is in their everyday life.
To that end, what is striking is not that "vehicle" is so closely associated with "automotive," but to spacecraft (in the 1950's), which people back then had little need to deal with in their everyday lives. (There is of course the objection that, in the wake of Sputnik, everyone was seized with a fascination on spacecraft. True, but that only goes to show how hard it is to develop proper factors for normalization, an essential task of good characterization.) The fact that, today, there are so many collocates of "vehicle" with drones is also significant, given that most people do not actually fly drones (yet) in their everyday life.
I do not propose actual normalization numbers now, but, however one chooses to normalize about how often people interact with "automotive" versus "aerospace" in their lives, it does seem that, across time, no one has hesitated to extend the use of "vehicle" to refer to "aerospace," whether spacecraft or drones. And, with this poor characterization, the authors have no ground to conclude that the numbers show that aircraft do not fall within the scope of "vehicles."
Posted by: anon | Sep 13, 2017 1:04:55 PM
"But if you want to go beyond that, to look how the term is *ordinarily* used, those methods can be problematic."
So let's say we want to do a corpus linguistics on the word "problematic"--we want to know how "problematic" was ordinarily used in the 1980s and in the 2010s. Would we expect that its used differently or just more often?
Posted by: The Problem Attic | Sep 12, 2017 9:57:16 PM
James Heilpern makes a good point when he says that it would be appropriate to use corpus linguistics when dealing with terms of art in fields other than law. However, I would add the qualification that in order to do full-blown corpus linguistics, you would need to have a corpus of texts from the relevant discipline. That would rule out using the BYU corpora, which are the only game in town for purposes of using corpus data in legal interpretation. It would be possible to use Google and other resources as a surrogate for a true linguistic corpus (I refer to that as doing "informal corpus linguistics"), and that would certainly be good enough if your aim is simply to show that a the term in question *can* be used in a certain way. But if you want to go beyond that, to look how the term is *ordinarily* used, those methods can be problematic.
And note that this kind of informal corpus linguistics might be useful in dealing even with legal terms of art. Assuming there is no authoritative declaration of what a term means, such as a statutory definition or court ruling, it might be worthwhile in some cases to look at how the term is used out in the wild.
Posted by: Neal Goldfarb | Sep 12, 2017 7:09:59 PM
For those who think that computational linguistics can be used to understand "terms of art," would you use it to write a letter to a client advising on the scope of "consent" in a rape statute? "interstate commerce" in a federal criminal statute? "blood alcohol content" in a drunk driving statute? I would hate to be the first to explain to a bar disciplinary panel why that is not malpractice. And the idea that statutes for ordinary folks should be drafted to be understandable to them is an aspiration. It is not, however, an operational principle.
Posted by: anon | Sep 12, 2017 6:48:12 PM
For those who think that computational linguistics can be used to understand "terms of art," would you use it to write a letter to a client advising on the scope of "consent" in a rape statute? "interstate commerce" in a federal criminal statute? "blood alcohol content" in a drunk driving statute? I would hate to be the first to explain to a bar disciplinary panel why that is not malpractice. And the idea that statutes for ordinary folks should be drafted to be understandable to them is an aspiration. It is not, however, an operational principle.
Posted by: anon | Sep 12, 2017 6:48:11 PM
Anon: I think we need to be careful with statements like "corpus linguistics is not suitable for interpreting 'terms of art.'" Jurists can be a bit sloppy and often do not distinguish between legal terms of art and technical terms of art. While I have not given much thought to the prior, I think CL certainly has a role to play in helping judges define the latter.
As Justice Frankfurter explained, "If a statute is written for ordinary folk, it would be arbitrary not to assume that Congress intended its words to be read with the minds of ordinary men," but "if they are addressed to specialists, they must be read by judges with the minds of the specialists." In other words, the threshold question in a technical term of art analysis seems to be who is the regulated community? If a statute regulates a specific industry, we are no longer interested in the ordinary meaning of the statute with respect to the public at large, but rather the ordinary meaning within the regulated community. Because judges are outsiders to this community, they treat the question of whether a term has acquired a specialized meaning within a given industry as a question of fact. It seems that empirical data drawn from a specialized corpus (of say, the Petroleum industry or Aviation industry) would be far more helpful to a judge than the testimony of an expert witness, who's testimony may be skewed by his own linguistic biases, or a specialized dictionary.
I'm currently writing an article on this subject that I hope to upload onto SSRN sometime in October.
Posted by: James Heilpern | Sep 12, 2017 3:44:03 PM
I'd like to thank Carissa for starting this interesting debate about corpus linguistics and legal interpretation. I don't really have a stake in the corpus linguistics debate, but I find some of the arguments against judicial use of corpus analysis to be quite interesting. I want to make just a couple of comments.
1) The "notice" issue has already been discussed but there is a tension between notice and the idea that the language in a typical criminal statute is not amenable to corpus analysis because it is full of terms of art. I don't think this claim is accurate (as a search of the number of cases invoking the "ordinary meaning" of language would confirm), but if so that would seriously undermine the idea of notice. Ordinary citizens would not be able to understand what the law criminalizes because the statutes would, essentially, be written in a foreign language.
2) I think some of the objections to corpus analysis ignore some of the features of legal interpretation. For instance, as the Costello case illustrates, judges habitually make general claims about the meanings of words. It’s an essential aspect of interpretation (both legal and non-legal). The Costello case is interesting because of the Google searches, but judges routinely rely on dictionary definitions (how they do so is seriously flawed, but that’s another debate). Why do they do so? Consider two possible interpretive universes:
a) The judge believes that interpretation is so context-specific that nothing general can be said about word meaning.
b) The judge believes in both semantics and pragmatics. In essence, the judge believes that words have general meanings that are, to various degrees, shaped by the context in which they are used.
In the first universe, I’m not sure how interpretation would work. The judge would have to forgo any serious consideration of the language (maybe ruling instead based on what the law should be) and decide cases by fiat: This is what the law means.
The second scenario is, of course, not easy. The judge must be careful not to exaggerate the determinacy of language (something textualists are often guilty of doing) and should recognize the existence of generality, vagueness, and ambiguity. The judge must also have ways of resolving linguistic indeterminacy. But if the judge recognizes that words have meanings that cut across contexts (even if there is a certain degree of indeterminacy), there must be some way of determining those meanings. Judges can, and do, use their own knowledge of language, but judges, naturally, desire external verification of their intuitions. Dictionaries are a natural source of meaning because they are neutral to the interpretive dispute and are created by experts. If a dictionary is a legitimate source of meaning, why is corpus analysis not? Is it that dictionaries capture ordinary meaning better than corpus analysis? That’s a difficult argument, I think. Of course, I’m leaving out the issue of whether judges are capable of performing a competent corpus analysis, which is a separate issue.
Posted by: Brian Slocum | Sep 12, 2017 2:52:56 PM
Thank you for your reply. I would submit that the identification of "terms of art" is not that simple, and can trip up even the greatest jurists. We all laugh now at Justice Cardozo's infamous quotation from the Bible, "life in all its fullness," but his creation of the "life in all its fullness" test is a direct result of his failure to recognize the word "ordinary" as a "term of art" in tax.
Posted by: anon | Sep 12, 2017 2:45:33 AM
anon asks, "how does one even draw a line between a "term of art" and one that is not?"
The answer is: By going back to what you learned in Legal Research 101.
If a word is indeed a term of art, there will be a record its being used in the relevant term-of-art sense. So start by looking in Black's to see if a specialized legal meaning is given. Then do search on Westlaw or Lexis. (Does "Words and Phrases" still exist? You could look there, too.) The main difficulty, I think, would be framing a search that would limit the results as much as possible to the term-of-art meaning, and that's probably not terribly difficult. Also, if you know the area of the law that the word's specialized meaning relates to, look in a treatise for that area.
Posted by: Neal Goldfarb | Sep 11, 2017 8:01:37 PM
From the other thread, it seems that there is some agreement that corpus linguistics is not suitable for interpreting "terms of art." I personally agree as well. So let us take it for granted then that it is only acceptable to use the methodology whenever the term to be interpreted is not a "term of art." But how does one even draw a line between a "term of art" and one that is not? As Kerr pointed out in the other thread, many statutes are copy-and-paste jobs. And when "battle-tested language" get recycled again and again, with the intention that the recycled bits keep the same meaning as before, how much is left that is not a "term of art"? One would think the remaining use cases are for words so commonly used that the words could not be deemed to consistently bear the same meaning in different statutes. But then, in these cases, why would the methodology, which is built to generate only one "best" interpretation, be of any use?
Posted by: anon | Sep 11, 2017 6:16:36 PM
"Some commenters suggested that the corpus database is more likely to mimic public intuitions about meaning because the public reads the newspapers and magazines that make up the corpus. But judges read those very same sources. Why would the corpus database be an accurate reflection of a non-judge’s linguistic intuition, but not a judge’s intuition? I don’t disagree with those who say that corpus linguistics databases capture more information about language usage than does a judge’s own status as an English speaker."
I think this misframes things. It isn't that judges' intuitions are especially dissonant with the corpora, while the corpora capture non-judges' intuitions. Rather, I think the CL claim is that if you want to arrive at the interpretation of a statute that best captures the population's understandings as a whole, you'll do better to survey all the usage you can than to ask a few people in a nation of hundreds of millions how they understand the language, on the basis of their exposure to a small fraction of the sources in the corpora. The claim isn't that an individual non-judge's intuitions are more likely to be consonant with the data from the corpora than a judge's; of course that's possible depending on what the judge reads and who he knows, but it's not the claim. Rather, the claim is that a huge sample of usage is a better measure of aggregate understandings than a few individuals' understandings, which are grounded in their much smaller half-remembered samplings of usage. I think that's pretty obviously correct, but I do think in practice the project tends to break down, and that the search for frequencies is misguided. People don't, I think, disambiguate ambiguous remarks, usually, by reasoning that a word is used more often to mean A than B, so odds are that it means A. There are so many confounding factors that make that an unwise approach. For example, Lawrence Solan has a paper arguing on the basis of CL that to defraud a bank you need to defraud them out of some money they really own; it doesn't suffice to defraud a bank into giving you one of their customers' money. (The Supreme Court unanimously disagreed.) Well, it may be that the reason "defraud a bank" is usually used to mean the former, not the latter, is that frauds of the latter kind are much rarer, not that people don't think of them as bank fraud.
Posted by: Asher Steinberg | Sep 11, 2017 5:37:03 PM
I am not sure if I should be commenting on this post or the previous post, so I am sorry if this is in the wrong place. I am a bit late to the discussion, but I hope my comments will be somewhat helpful.
Having written an article discussing the application of Corpus Linguistics to criminal law and specifically to the use of the rule of lenity, I want to push back on the suggestion that corpus linguistics is inappropriate. Reading your comments above, and the article, I see two primary concerns, though I am sure there are others, that I want to respond to briefly.
First, the suggestion that use of a corpus is contrary to public notice because it involves the usage of a tool unfamiliar to legislatures or the general public. As other commentators such as Stephen Mouritsen noted on the original post, the use of a corpus is primarily a tool for figuring out what an ordinary person encountering certain words in a statute would understand them to mean. We interpret words and phrases based on our lived experience with those words and phrases. Corpus linguistics is merely a tool to show what experiences one would have had with those words. That seems especially appropriate to me in the criminal law context where notice is a matter of fundamental due process. When interpreting criminal statutes it seems ever more important to consider what the criminal defendant would have understood those words to mean based on how those words are commonly used in the culture around him. With criminal statutes it seems far less significant to me to ask what Congress intended the act to do, and far more significant to ask what would someone of ordinary experience encountering those words have understood. In other words, even if Congress really really want to proscribe certain conduct, if their words would not have conveyed that to the average person, then the statute would not provide adequate notice. So corpus analysis seems more rather than less appropriate.
(That raises an interesting question that I suspect could merit a fuller exploration: When interpreting older criminal statutes is it appropriate to consider what the statute meant at the time of enactment or at the time when the criminal defendant committed the act. And how might Corpus Linguistics aid in that endeavor).
Second, the suggestion that corpus linguistics supplants or replaces the role of a judge in making substantive decisions and determinations about the meaning of law, and that doing so is wholly inappropriate in the criminal context. It does not appear to me that corpus linguistics can ever really supplant or replace other tools of statutory interpretation, and that includes substantive canons such as the rule of lenity. I wrote my article on the rule of lenity that you cite in your article precisely because I think corpus linguistics furthers the use of that canon of interpretation. In my view, the Rule of Lenity is a strong thumb on the scale of interpretation that means that a defense favoring interpretation will be chosen even if it is not the most common or widely used interpretation of a statute. The data produced from corpus linguistics helps to further application of the rule of lenity in two ways. First, it makes it clear that a defense favoring usage is not only theoretically possible, but actually attested to in real usage of English. That gives far greater strength to a defendant's argument that the statute could permissibly be read in a certain light. Second, it provides some measure of evidence as to how frequent or common the defense favoring interpretation is vis-a-via the prosecution favoring interpretation. As I mention in my article, various members of the Supreme Court have employed dramatically different metrics for when the rule of lenity triggers. The use of a corpus does not supplant the need to make the substantive determination of what exactly is required for something to trigger the rule of lenity. Nor does it completely take away a judge's ability to make case-by-case determinations with regard to where that standard should be located with any particular statute. For instance, it might be reasonable for a judge to require a lesser degree of ambiguity before lenity is triggered when a statute imposes a particularly harsh penalty or is particularly likely to be deceptive or difficult for the ordinary public to follow. Corpus data can be useful regard of how strictly you interpret the rule of lenity. It does not supplant the need for thoughtful judicial interpretation.
Posted by: Daniel Ortner | Sep 11, 2017 4:59:38 PM
Hi Neal:
I'm always happy to have people disagree with me--it means I get to talk more :) And I am definitely looking forward to your blog post.
In asking whether judges should be permitted to consider corpus linguistics evidence when interpreting a statute, I think that you are getting at my first point: whether corpus linguistics is just a tool, or whether it is a method of statutory interpretation. I appreciate that you and some others who champion corpus linguistics do so as a tool, rather than as an interpretive methodology. But I read Lee and Mouritsen as advocating for a new interpretive approach, rather than simply a new tool. It is the interpretive approach that I am concerned about.
While I haven't given it much thought, I guess I'm agnostic about the types of evidence that the parties are permitted to introduce in support of their proposed statutory meaning. My concern is about the weight that courts give that evidence and whether they give certain evidence priority over other interpretive rules. My particular concern is that courts will use corpus linguistics evidence to avoid lenity and clear statement rules.
I hope that answers your question!
Posted by: CBHessick | Sep 11, 2017 4:15:03 PM
Carissa, I'm glad to see that you love to disagree with people, because that means (I assume) that you also love to have people disagree with you. If that's the case, you'll really like my post at LAWnLinguistics once it's up.
In the meantime, a few questions:
Given that your objection to the use of corpus linguistics is structural, does that mean that if the defendant in a criminal case wants to make a corpus-based argument, the court shouldn't consider it?
One of the problems you see with using corpus linguistics is that it relieves the judge of personal responsibility -- presumably because the judge can say, "I'm just following what the corpus data shows." Does that mean that you are totally discounting the possibility that judges could consider and weigh corpus evidence, just as they consider and weigh other evidence of meaning, and give the corpus data whatever weight and significance they think it deserves? And if so, aren't you saying that because corpus analysis might be used inappropriately in some cases, it should never be used?
Posted by: Neal Goldfarb | Sep 11, 2017 2:56:33 PM
The comments to this entry are closed.