« Transplant Tourism: Hard Questions Posed by the International and Illicit Market for Kidneys | Main | On Policy and Plain Meaning in Copyright Law »

Wednesday, April 24, 2013

Some Thoughts on Counting Terror, and Counting in General

This map of terror incidents across time and space in the United States (discussed more in detail here) is simultaneously fascinating and mind-boggling:

Screen Shot 2013-04-24 at 12.45.54 PM
(This is only a screenshot: my efforts to embed the video itself failed miserably.)

It is impossible to look at this map and not immediately ask: “What exactly is a terror attack?” According to the research group at the University of Maryland that produced the map, “terror” is: 

The threatened or actual use of illegal force and violence by a non-state actor to attain a political, economic, religious, or social goal through fear, coercion, or intimidation….

Now that obviously can’t be entirely true. By this definition, every drive-by gang shooting would be an act of terror: a non-state actor using intimidation to secure economic gains. Arguably any act of domestic abuse, particularly in a semi-public or public place, could likewise be seen as an act of terror, since it represents an effort to use fear to advance a social goal (keeping women in a second-tier position). And so on. Simple armed robbery? Seems to fit. 

I think this map provides a clear example of a problem all too rarely discussed in statistics and the quantitative social sciences: no matter how hard we try make our models “objective,” at the end of the day the numbers we use in them are almost always quite subjective. Their "numberness" makes them seem objective, but they are not. And we need to either (1) openly note the underlying political/legal/moral assumptions that drive our numbers or—and this is my preferred option—(2) attempt to model more explicitly how our results vary with different political/legal/moral assumptions. 

In other words, there is no “objective” definition of terror, thus any measure of it we make will always be inherently political. If nothing else, note that the federal government uses a different definition than the one that undergirds this map: most tellingly, the federal definition drops “economic” from the justifications. Change the definition, change the results.

We see this in lots of situations. Illinois, for example, did not provide rape statistics to the Uniform Crime Reports for years because Illinois has used a different, broader definition of rape than the federal government. Or think about poverty: an income of $11,490 is poverty for one person, $15,510 for a family of two, and so on by $4,020 increments up the chain. That can’t be objectively “true”: why not increments of $5,020? Why not a baseline of $13,000?* Even murder: if you’re counting murder, should you count cases where someone kills another in self defense? Ostensibly, the person is still guilty of murder but has an affirmative defense. It may make sense to exclude that from a study of what-we-think-of-as-murder, but that’s a normative call.

 Yet social scientists almost never wrestle with these issues. We use the UCR’s rape or murder counts without asking if they really measure what we care about. And we insert “percent in poverty” into regressions all the time without carefully thinking about the host of normative and political decisions that go into that percentage and, more important, without thinking about whether those explicit and implicit assumptions are consistent with the research projects we are engaged in.

Sometimes our hands are tied: whoever gathers the statistics gathers them in a particular way, and it would be prohibitive to regather it ourselves. If the UCR excludes justifiable homicides from its murder stats, it is almost impossible for me to go back and collect that data myself. But sometimes it is laziness: poverty statistics are calculated from the CPS, and that is freely downloadable. I could compute my own poverty percentages if I wanted, but it would take time and energy, but it would force me to think more critically about what exactly I am measuring.

Yet even if it is impractical or inconvenient to recalculate the data, there are still steps we can take to better acknowledge the inherently normative content of the numbers we use. The first, and to me less appealing, appoach is to simply be more open about exactly what normative assumptions we—or the third party who gathered the data—are making when we arrive at the numbers in the data, and to defend those assumption.

But if the definition is ultimately normative or political, it really can’t be defended in any objective way: we choose it because we believe it, but we can’t “prove” that we’re right. Just like a utilitarian can’t “prove” a retributivist wrong and vice versa. That's what makes a "political" or "normative" issue political or normative in the first place. 

So the second, more appealing, approach is to model much more explicitly what turns on various political choices. Take the terror map. I would have loved to see three maps in a column, each showing the progression of terror attacks over time in sync, but using different definitions of terror. This would illuminate just how important the normative calls are, or perhaps for what questions they matter.

Stick with the terror example. Assume we all agree that Oklahoma City, 9/11, and a few other major events count as “terrorism.” Then it could be that our definition matters a lot for the number of terrorist events but is relatively unimportant for measuring the total number of terror-related deaths (since the clear-cut cases are few in number but dominate when it comes to human cost). So depending on what you are trying to count, the choice of definiton may be more or less important. That's really useful—essential—to know.

Yet we so rarely see this. The dominant approach is to pick a definition, perhaps—but generally not—defend it, and then treat it as the “right” definition, or at least the “best we can do” definition. But the fact that this is standard practice doesn’t mean it is the right way to go; it clearly isn’t. We need to acknowledge more openly the inherently political and normative nature of what we do, and we need to model much more honestly just how much turns on these normative and political choices.

And consumers of empirical work need to take care not to view the numbers in an empirical model as being perfectly objective measures of the phenomenon under question, at least most of the time. Perhaps if we want to measure “the total number of people in the room,” then an enumeration pulls that off in a purely objective way: either there are 10 people in the room or 11, and I don’t think we’ll fight over how to define “people” (but if I had said “adults”…?). But note that if we want “the total number of people in the country,” a number equally objective in theory and one that lacks the normative ambiguity of “poverty” or “murder” or “terror,” we face the fact that the decision to count rather than sample will produce different estimates. Unacknowledged subjectively lurks in almost every model; sometimes it matters, sometimes it doesn't, but the fact that it is never discussed is deeply troubling.


* Even worse, the poverty scheme was developed in the 1960s, and the only adjustment made to it since then is for inflation. So the standard seems likely to be out-of-date.




Posted by John Pfaff on April 24, 2013 at 01:00 PM | Permalink


TrackBack URL for this entry:

Listed below are links to weblogs that reference Some Thoughts on Counting Terror, and Counting in General:


John, I'm not a moral philosopher, although I do read works in ethics and moral philosophy. And I'm not claiming we cannot distinguish between state and non-state terrorism, only wanting to stress the fact that they are both, for moral, political, and legal purposes, species of terrorism. Terrorism is difficult to define, be it of state or non-state origin, and it is frequently a "slippery and much abused" term. Yet "terrorism" is not a purely subjective concept, however much we notice its "shifting and contested meanings" or the fact that beyond its literal signification, it has the power to "stigmatize, delegitimize, denigrate, and dehumanize those at whom it is directed...." Moreover, it is certainly true that the term is often "ideologically and politically loaded," implying moral, social, and value judgments. Frankly, I have no idea what a "moral-relativist economist" is, although I do know a bit about the notion of moral relativism and believe it not to be true. (At one level, social 'mores' and/or moral systems will of course differ, but at bottom, owing to human needs or human nature, and/or to our learned capacity for social cooperation and social life, any conceivable moral system, in the words of P.F. Strawson, 'will include the abstract virtue of justice, some form of obligation to mutual aid and to mutual abstention of injury, and, in some form or degree, the virtue of honest.')

I can assure you that more than a few contemporary moral philosophers (like C.A.J. Coady or Larry May) would view the bombing of Dresden, the firebombing of Japanese cities and the dropping of the atomic bombs as exemplary instances of state terrorism, although we can of course find far more recent examples across the globe, including our war in Indochina and the following from a recent Congressional Research Service report:

"Cluster munitions were used by the Soviets in Afghanistan, by the British in the Falklands, by the Coalition in the Gulf War, and by the warring factions in Yugoslavia. In Kosovo and Yugoslavia in 1999, NATO forces dropped 1,765 cluster bombs containing approximately 295,000 submunitions. From 2001 through 2002, the United States dropped 1,228 cluster bombs containing 248,056 submunitions in Afghanistan, and U.S. and British forces used almost 13,000 cluster munitions containing an estimated 1.8 million to 2 million submunitions during the first three weeks of combat in Iraq in 2003."

In addition, as Robert Scheer informs us, "Israel is said to have dropped almost 1 million unexploded bomblets in Lebanon in the 2006 war against Hezbollah, which fired 113 cluster bombs filled with thousands of bomblets at targets in northern Israel."

Alas, acts of state terrorism are found common to both democratic and non-democratic regimes.

As to moral and legal definitions of terrorism, I would recommend Ben Saul's Defining Terrorism in International Law (2006), and several of the references in the SEP entry on Terrorism, including works by Coady.

Normative enterprises need not be subjective and the moral criteria used to identify and assess acts of state terrorism are fairly clear and straightforward provided one makes a sincere attempt to be impartial and objective, as moral (and political) philosophers endeavor to accomplish as part of their vocation.

Posted by: Patrick S. O'Donnell | Apr 24, 2013 5:50:15 PM

@PSO'D: Actually, that's my point. There is a perfectly legitimate research project to measure non-state-actor terrorism. State terror and non-state terror are two different things, and sometimes it makes sense to study them together and sometimes not. To lump them together is a political decision, and simply asserting that it is immoral to separate them out doesn't actually make it so.

And think about how hard it is to define "state terrorism," even if I accepted your point. Just to pick one example, WWII was basically 70 years ago, and we still can't decide if Dresden and the firebombing of Japan constituted terror or not. Because subjective concepts like "terror" have no precise meaning. How we count incidences of state terror will still be a highly subjective, normative enterprise.

But I'm an moral-relativist economist, and you're a moral philosopher. So it's not surprising that we disagree about such things.

Posted by: John Pfaff | Apr 24, 2013 4:53:47 PM

Any definition of terrorism that excludes State terrorism is morally, politically, and legally indefensible.

Posted by: Patrick S. O'Donnell | Apr 24, 2013 4:19:09 PM

As an example, Columbine is on the list, but not many workplace shootings, or even other school shootings as far as I can tell.

Posted by: Bruce Boyden | Apr 24, 2013 2:24:04 PM

The comments to this entry are closed.