« Conscience, Speech and TIAA-Cref Advice | Main | Advice for Getting a Job on the Law Teaching Market »

Friday, May 22, 2009

Actuarial Criminal Policy

A student of mine sent me this article from the Wall Street Journal about efforts in LA to develop an actuarial model to determine what children are most at risk of joining a criminal street gang. It is exciting to see state officials say things such as this:

Previously, city officials depended on what they concede was a patchwork of information to build gang-prevention programs, often using anecdotal tips from local beat cops or high school teachers. "We were not relying on data," says Rev. Jeff Carr, an evangelical minister who is the city's "gang czar," leading outreach and prevention efforts. "We had gang-prevention programs, but no criteria to determine who was in a gang."

The article also points to a silver lining of the current economic downturn as well. With state resources drying up--a particular problem in that financial basket-case called California--the need to allocate resources carefully grows. As the article continues:

In Los Angeles, Dr. Klein's theories are appealing to policy makers eager to stretch limited resources. This year, the test is being given to children for the first time, and officials say they will use the results to determine whether some of the city's $24 million annual budget for gang prevention is being spent on children who aren't at high risk.

Of course, identifying who is a gang member is only the first step; the next is to develop effective interventions. Here, however, the article raises a big red flag. Describing current efforts, the article states:

So far, 958 children who live in active gang areas have taken the test; of that group, about one-third have been identified as potential future gang members and will be enrolled in prevention programs. But city officials won't know for several years whether the test failed to pick out children who went on to join a gang.

It looks like all 958 have been enrolled in programs. So there is no control group to see if the intervention works. Ideally, a random sample of those testing "positive" should not be enrolled to see what effect the program has. Such randomization is rare, particularly in criminal policy, but not impossible. Without such randomization, it will be hard to know how effective the interventions are (unless, perhaps, the goal is simply to compare the relative effectiveness of different programs against each other, rather than against an absolute bar of "effective or not," although even then I would like a control group). 

This failure to randomize raises another problem. In the future, the police will be able to identify the false negatives--those not classified as potential gang members but who later join gangs--but not false positives. With all at-risk people committed to treatment, it becomes much harder to know if those who later do not join gangs fail to do so because of the treatment or because they were false positives.

But these design flaws aside, the program in general represents a positive step forward, of moving past anecdote and intuition towards something more rigorous. 

Posted by John Pfaff on May 22, 2009 at 09:25 AM | Permalink


TrackBack URL for this entry:

Listed below are links to weblogs that reference Actuarial Criminal Policy:


Feed You can follow this conversation by subscribing to the comment feed for this post.

Looks like Los Angeles is in need of meta-analysis. There is a vast criminology literature out there, much of it pertinent. While testing your wheel design has virtue, there is no need to try a square or triangular one, when round is proven to be effective.

One particularly strong predictor of future criminal activity (in addition to male gender), for example, is failure to graduate from high school. In Colorado, a high school dropout is 3.2 times as likely to become a prison inmate as someone who finishes high school but doesn't earn a college degree, while someone who earns a college degree is about 50 times less likely to become a prison inmate than someone who finishes high school but doesn't earn a college degree. A high school dropout is roughly 160 times as likely to become a prison inmate as someone who earns a college degree. A male high school dropout is more than 1500 times as likely to become a prison inmate as a female with a college degree.

Dropout propensity, in turn, is strongly predicted by flunking high school courses, and by high absence levels, below grade level test scores, and incidences of school discipline at least as early a middle school. (Even dropping out of high school for seemingly personal reasons like pregnancy is surprisingly closely linked to absence levels and test scores.)

One hot off the presses study in Denver, for example, found that: In ninth grade, most of the dropouts had gotten at least one F in their freshman year, a third had four or more F's in a semester, and two-thirds of the dropouts had missed 20 or more days of school in the ninth grade. When they were in sixth grade, one third of the dropouts were failing at least one course, 44 percent had missed more than 20 days in middle school, and one in five had at least one suspension in middle school.

Who are your future criminals? The ones who in sixth, seventh or eighth or ninth grade have a couple dozen unexcused absences, who aren't at grade level for reading, writing and math on school assessments, who have flunked classes, and who have been suspended from school. (In practice, these multiple factors overlap heavily, so weighting isn't as important as it might seem a priori.) Future criminals aren't necessarily future gang members, but the overlap is great.

These factors developed in prior studies, moreover, have the virtue of being capable of being assessed with small quantities of quantifiable, highly summarized information already maintained for other purposes in school administrative files. In a typical school, an "at risk" list could be developed in a couple of afternoons with a couple of staffers, but benefit from a lot of meta-analysis validation from prior studies in other places.

For even better accuracy, one could run juvenile records checks on youths identified from school records to be at high risk, since recidivism rates are high at all levels of the criminal justice system.

The hard part is not identifying the kids at risk, there is lots of good evidence on that score. The hard part is finding programs that have been shown to work to turn them around. Records on this score are often mixed with similar programs in different places having very different success rates and measures at different times.

Posted by: ohwilleke | May 29, 2009 1:27:41 AM

Post a comment