A Flawed NRC Report: Prison Populations and Sentence Length, Part 2

(This is Part 3 in my criticism of the recent National Research Council report on incarceration. Here are Part 1 and Part 2.) 

As I noted in my previous post, the NRC report's focus on longer sentence length failed to address multiple papers--admittedly written by me, so not exactly a neutral observer here--that provide at least credible empirical results that longer sentence length really isn't that important. Here I want to turn to problems with the results the NRC itself relies on. That the report ignored dissenting positions is made all the worse by the fact that its own analysis is quite weak.

To compute how much time inmates serve, the report simply takes the total stock of prisons and divides by the number of admissions. So if in 2000 there are 1,000 people in prison for murder and 100 are admitted for murder, the average time served in 10 years (1000/100). There are many problems with this.

1. As the report acknowledges, this is an accurate measure only if the system is in equilibrium, which it never is here. The report tries to adjust for this a bit, but the adjustments are unconvincing.

2. Why do they only report the average, which by definition cannot be calculated for most admission cohorts (since we don’t know how long those unreleased will spend in prison until released)? Where is the median (which can be calculated), and other quintiles? This isn’t just a question of data availability: it useful to know how skewed the prison population is, but the report does none of this (even though there is a paper out there that does).

3. As my papers show, there is another source of data, the National Corrections Reporting Program, that allows you to look at individual-level data to measure more exactly just how long inmates are staying in prison.

4. This next criticism is the most technical, but it perhaps the most important, because it points to the very sort of careful technical analysis NRC reports are supposed to do but which this one completely failed to do. The problem with the NCRP, one could say, is that it applies to only a limited number of states, and the NRC report relies on national-level data from the BJS on incarcerations by offense type. But how does the BJS get that national-level data? By extrapolating from the limited data in the NCRP. And for technical reasons that I have written about extensively but which the committee again ignored, there are reasons to be concerned about using national-level aggregates from the NCRP. This is the very sort of really-understanding-the-data issue that we should count on NRC reports to explore (see the exemplary one on the death penalty), but there is none of that here.

In other words, even if the criticism in point 4 seems obtuse or unclear, it should raise a red flag for everyone reading the report: it indicates that it wasn’t thinking carefully about the data, a cardinal sin in statistics. 

In fact, I can go further. The NRC report claims that the biggest increase in sentence length was for murder, which rose to 17 years by 2000. Let’s see how plausible that is, using state-specific NCRP data. I looked at data on murderers (both murder and manslaughter) admitted in 2000 in New York, Missouri, and Georgia. I have data on how many are released each year between 2000 and 2012. In New York and Missouri, two-thirds of all murders are released by 2012, and in Georgia (more Southern, more red, more punitive) almost half. (In NY the median time to release for a killer is 6.5 years, in Missouri 4.5 years.)

So given the number of people admitted, the time served by those released, and the number of people remaining in prison, it is easy (grade-school algebra is actually useful in real life!) to calculate how long those admitted in 2000 and still in prison at the start of 2013 have to stay in prison for the average time served for by all murderers admitted in 2000 to be 17 years. For New York, the 341 remaining killers (out of 1057 admitted in 2000) would have to serve an average of 44 years; for Missouri, the remaining 121 (out of 363) would have to serve 44 years as well; for Georgia, the remaining 228 (out of 453) would have to serve 29.

The numbers for New York and Missouri are implausible. First, that’s the average, so a significant number would have to serve more than that, and at that point you’re hitting the actuarial limit for being alive (since this is time actually served, not sentence imposed). Georgia is more credible, but given the number still in still unlikely. 

Again, these are just some quick back-of-the-envelope calculations, but they provide an important counter-arguments to the crude calculations used in the NRC report--and I should point out here that the report bases its entire conclusion on a single unpublished, un-peer reviewed presentation given to the committee by Alfred Blumstein and Allan Beck.* It is possible to argue that my murder-plus-manslaughter elision is too important to overlook, or that 30 or 40 year sentences are not as implausible as I think. These are all valid concerns to raise. But this is a blog post, not a National Research Council report. I put this together in about two hours, not the months and months of time the committee had. 

So not only does the NRC report overlook rigorous studies reaching different conclusions, but its own analysis relies on a crude approximation that weakens signficantly when subjected to close scrutiny using the very dataset it implicitly relies on.

An NRC report provides a valuable--and rare--opportunity to closely analyses controversial, policy-relevant data with a rigorous eye. As should be clear by now, whatever the right anwer, this report has simply failed to do so.

 

* In a future post, I will discuss more fully the committee’s surprisingly naive, unrigorous use of empirical work in report. It stands in stark contrast to other NRC reports, such as that on the death penalty, which wrestled with deep methodological issues openly and with sophistication.