« The truth about past relationships | Main | "Constitution USA" with Peter Sagal »

Tuesday, May 07, 2013

Some More Thoughts on Stop and Frisk I: Comparative Baselines

In light of some of the comments on my previous post about stop and frisk, I thought I’d say a few more things about it. In this post I want to focus on Larry Rosenthal’s claim that because crime dropped at the same time that the NYPD adopted stop-and-frisk hotspot policing, we should be wary about dropping stop and frisk.

The core problem with this argument is that it treats stop-and-frisk-of-crime-hotspots as a single concept. But there are two distinct ideas here: hotspot policing does not require stop-and-frisk. Is it the hotspot part that matters, the stop-and-frisk part, or some combination of both? Admittedly, the simultaneous timing poses a serious problem for disentangling which matters more, but it may be possible to glean some rough conclusions.

Conveniently, David Weisbrud—recipient of the Stockholm Prize, criminology’s answer to the Nobel, for his work on (among other things) hot spot policing—has a recent article titled Could Innovations in Policing have Contributed to the New York City Crime Drop even in a Period of Declining Police Strength?: The Case of Stop, Question and Frisk [SQF] as a Hot Spots Policing Strategy (written with Cody Telep and Brian Lawton). Almost too on-point to be believable. The main conclusions are quite apt to the discussion in the comments. Ultimately, Weisbrud et al. come down in favor of the hotspotting, but not the stopping and frisking.

First, their pro-SQF claims:

These data suggest that SQFs are applied in New York as a focused hot spots policing strategy. In turn, … even during the period of a strong decline in the number of police, the NYPD was able to increase the level of SQF activities significantly. We think it is a reasonable conclusion that New York City was able to “do more with less” in terms of its hot spots policing approaches. Despite decreasing strength in the department overall, Operation Impact allowed the NYPD to put increasing focus on SQFs as a hotspots policing strategy.

But then they make this important caveat, which I want to quote in full (added emphases are mine):

We want to emphasize that our comments here regarding SQFs are not an endorsement of the approach relative to other hot spots strategies. Given the possible negative impacts of SQF policing, both on citizens who live in such areas, and the primarily young and minority population that is the main subject of SQFs, we suspect especially in the long run that this approach will lead to unintended negative consequences. In particular, if negative encounters between police and residents lead to lowered perceptions of police legitimacy (Tyler, 2004), then any gains from aggressive police efforts in the short term could lead to long-term increases in offending.

This is especially relevant to New York City, where some have argued that aggressive police tactics in the 1990s damaged police–community relations even before the rise of SQFs (e.g. Greene, 1999; Harcourt, 2001). Concerns about police legitimacy are particularly salient in minority communities, where views of the police are often very negative. Weitzer and Tuch (2006), for example, found in their national survey of 1,791 adults that almost 40% of black respondents said they had been treated unfairly by police because of race compared to just 2% of whites surveyed. Half of white respondents reported being very satisfied with the police compared to just 22% of black respondents. Because any hot spots approach is vulnerable to potential negative consequences for police legitimacy (e.g. see Kochel, 2011; Rosenbaum, 2006), Braga and Weisburd (2010) suggest that hot spots policing should be carried out with attention to procedural justice and concerns with police legitimacy. This in our view would be a more productive way to implement hot spots approaches.

Accordingly, while the NYPD appears to have capitalized on its Operation Impact program to increase hot spots policing approaches, we think that the same gains could have been achieved with potentially fewer long-term negative outcomes. There is no evidence that the NYPD has emphasized procedural justice and legitimacy evaluations in its efforts to police crime hot spots. Such emphasis is critical if programs like Operation Impact are to have not only short term, but also long-term crime control gains.

Weisburd et al. conclude by pointing out that despite Commissioner Kelly’s claim that his commitment to SQF was unchanged, district commanders appeared to be cutting back sharply in the wake of widespread public disapproval.

Their point is an important one: the relevant comparison isn’t between SQF and nothing, but between SQF and the next best alternative. Larry is inarguably right to point out in the comments that crime is highly concentrated, and that crime (and thus the savings from reduced crime) fall disproportionately on blacks. But the fact that SQFs in hotspots appear to “work” does not mean such an approach is efficient. And the legitimacy literature strongly suggests that more legitimate forms of hotspotting would likely be more effective in the long run.

The failure to pay attention to comparative baselines is a rampant problem in criminal justice work, and elsewhere.1 Large-scale incarceration, for example, “worked,” in that it surely contributed to the decline in crime since 1991 (and perhaps restrained crime growth during the 1960-1991 surge). But it also appears that a dollar spent on police may be 20% more effective than a dollar spent on incarceration. If true, we could have the same level of crime for billions less just (“just”) by shifting our resources from prisons to police.

All of which is to say that when policymakers adopt policies X and Y at the same time, simply observing that we achieved the desired outcome does not mean than Y, or even (X, Y), was the right policy to adopt. In this case, the compelling but often-overlooked (or at least underappreciated) legitimacy literature provides some strong, albeit by no means dispositive, evidence that SQF-Hotspotting was not the optimal bundle of policies.


1This is now a significant problem in medicine. Pharma companies are just required to compare their drugs to a non-pharma placebo, not to the current best option available. So they can market drugs that provide no real gains as “working,” since they are better than nothing, even though no one actually uses “nothing” in the relevant situation.


Posted by John Pfaff on May 7, 2013 at 09:11 AM | Permalink


TrackBack URL for this entry:

Listed below are links to weblogs that reference Some More Thoughts on Stop and Frisk I: Comparative Baselines:


In my view, the "core problem" here is that the post presumes there is evidence that hot-spot policing that eschews stop-and-frisk is likely to be as efficacious as stop-and frisk. Weisburd does not make that claim, and I know of no criminological literature to support it. Although there have been plenty of studies -- including Weisburd -- that demonstrate the efficacy of stop-and-frisk policing directed at hot spots, studies focusing examining increasing the frequency of patrol have not produced the same results. This should not be surprising -- officers on passive patrol have a limited ability to intervene effectively in the community. All you need is a lookout, and it is easy to insulate gang and drug activity from passive patrol. Stop-and-frisk, however, makes it much riskier to carry guns and drugs in public places. When guns and drugs move off the streets, in turn, the risk of violent confrontations declines.

As for Weisburd's caveat, my concern is with the lack of empirical evidence to support the speculation that aggressive policing will reduce community perceptions of police legitimacy and thereby stimulate "long-term increases in offending." People have been speculating that New York's strategy will have this effect since the late 1990s, as Weisburd's citations indicate. And yet, crime has remained low in New York to the present. My reaction is something akin to Paul Krugman's astonishment that those who claim that budget deficits and fiscal stimulus will produce inflation continue to be believed despite years of evidence to the contrary. How many more years do we have to wait before we can agree that New York has experienced "long-term crime control gains"?

Interestingly, Weitzer and Tuch, in the study cited, like other similar studies, found that community residents perceptions of local police are not statistically related to race, but rather to local crime rates. In other words, once one controls for local crime rates, the supposed relationship between race and perceptions of local police disappears. Perhaps the thing that really compromises police legitimacy is when police cannot keep community residents safe.

Larry Rosenthal
Chapman University School of Law

Posted by: Larry Rosenthal | May 7, 2013 1:03:47 PM

But that evidence does appear to be out there. Policing policy is not my primary area of empirical work, but a quick Google Scholar search turned up http://www.tandfonline.com/doi/abs/10.1080/07418825.2012.710645, which includes the following:

"A recently conducted hot spots experiment in Jacksonville, Florida (Taylor, Koper, & Woods, 2011) was the first to compare multiple hot spot treatments in the same study, with one treatment group receiving a saturation/directed patrol response and the second receiving a problem-oriented response. Results showed a decrease in crime (though not a statistically significant decrease) in the saturation patrol hot spots, but this decrease lasted only during the intervention period and disappeared quickly thereafter. In the POP hot spots, there was no significant crime decline during the intervention period, but in the 90 days after the experiment, street violence declined a statistically significant 33%."

Now "saturated patrols" may not be SQF, but likely they are closer analogs than POP responses. Admittedly, this is just one study, but it does indicate that the results are more muddled than you imply.

Also, in Braga's Campbell Collaboration systematic review, he states that:

"Unfortunately, the results of this review provide criminal justice policy makers and practitioners with little insight on what types of policing strategies are most preferable in controlling crime hot spots. Clearly, the enforcement-oriented strategies reviewed here work in preventing crime. We do not know, however, which enforcement strategies are more effective in preventing crime and under what circumstances are certain strategies more appropriate. This review also offers little insight on the effectiveness of enforcement tactics relative to other broader-based community problem-solving policing programs...."

If the evidence were so lop-sided, I would expect that Braga's SR would have uncovered some sort of measurable effect of SQF tactics.

Two last thoughts:

1. No one is disagreeing that NYC experienced a systematic change in crime rates, and one that can't just be explained as regression to the mean. And I believe most people would argue that policing played a role in that. But there is a counter-factual question at play: could the crime drop be even greater, or sustained at an even lower cost, if the police made some tactical changes? To say that to disagree with SQF is to somehow try to argue that police played no role or that NYC did not experience a fundamental shift is a strawman.

2. It's somewhat ironic that we're debating this, since my priors are probably closer to yours than our back-and-forth here would suggest. I believe that legitimacy and crime levels are clearly endogenous, and that the views of blacks in high-crime areas towards things like SQF are much more nuanced and ambivalent than that of those who tend to live in more privileged areas. Anecdotally, I've seen it myself at a local precinct/community meeting here in Brooklyn, when the working-class African-Americans who made up most of the attendees gave an anti-drug task force a standing ovation for a SQF and subsequent chase that got several guns off the street.

I think my resistance comes from the fact that you're arguing we *need* X *and* Y in the presence of what is for NYC an almost-perfectly colinear relationship, and in a setting where there do not appear to be any good RCTs elsewhere designed to isolate the effect of tactics-at-the-hotspot (rather than just being at the hotspot in the first place). Maybe SQF is required. But in the absence of rigorous evidence that it is needed, and given that there is evidence it may not be (or at least has crossed some sort of diminishing returns point--like Jeff Fagan's work showing that total guns seized does not seem to vary with the number of stops in recent years), and along with the important-if-ambiguous legitimacy concerns, it just isn't so clear to me.

All of which is to say I'm not arguing that SQF does or doesn't work, only that we know little enough about what is going on with it in NYC that we should be cautious about giving (or denying) it credit for NYC's (indisputable-by-all) crime drop. Feels like a "knowable but currently known unknown" situation.

Posted by: John Pfaff | May 7, 2013 2:12:32 PM

"In light of some of the comments on my previous post about stop and frisk"

John, you might want to note that the embedded URL in "previous post" points to the first of your "what is a drug crime" posts instead of a prior post on stop/frisk...

I think instead you meant the URL to point to the "Statistical Shenanigans" post?

Posted by: ?? | May 7, 2013 5:53:45 PM

“Stop and Frisk” is illegal behavior on the part of law enforcement and is a breach of civil rights for anyone stopped, regardless of their race. The actions and abuse by the NYPD are filling the very definition of a “Police State” where citizens are under never ending scrutiny in order for cops meet a quota designed to turn profits. You can read much more about our Justice System running amuck and how they’ve violated civil liberties across the country at http://dregstudiosart.blogspot.com/2012/08/the-privatized-police-state.html

Posted by: Brandt Hardin | May 8, 2013 2:52:44 PM

Post a comment