Thursday, July 10, 2008
Do We Need to Further Incentivize Scholarship, not Education?
In prior posts, I've been trying to figure out how to fill out the U.S. News survey of academics. I'm inclined to think we should be assessing the quality of a school's education, or more broadly, the "value added" for students training to practice law.
My impression, though, is that this is a minority view in the legal academy. The dominant view is that we should rank schools based on scholarship, using, say, Brian Leiter's rankings. Leiter's argument seems to be that we should be evaluating scholarly and educational excellence, and the former serves as a proxy for the latter. So by using faculty citations, we get a good, if necessarily imperfect, sense of what we're after. I disagree.
I've found one other argument for using faculty scholarship, from Russell Korobkin at UCLA, who was tapped to give the keynote address at the Indiana symposium on rankings. Korobkin had written a 1998 essay on the topic, and when Leiter did his first comprehensive study of the "academic distinction of law faculties" in the late 1990s, he said the study was undertaken in a "Korobkinesque spirit." (Leiter, Measuring the Academic Distinction of Law Faculties, 29 J. of Legal Studies 451, 454 (2000))
Korobkin's argument is basically this: rankings serves a "coordination function" between students and employers. Employers look to the rankings to figure out how to find the best future lawyers; prospective students look to the rankings to see where the most desirable employers will hire from. So it actually doesn't matter what the rankings are based on.
We could, for example, fill out the U.S. news survey according to The Sporting News's preseason college football poll -- and, hey, look at that, Georgia is #1! So the best students would start going to Georgia, the most desirable employers would focus their hiring there, and so on (not clear how much we'd increase our Supreme Court clerkships, as we've had four in the last five years, but anyway).
For Korobkin, the mere existence of rankings serves its function; its content is irrelevant to achieving the "coordination function." This seems like an important insight, and I have no quarrel with it.
It's the next step that is the weak link. Korobkin argues that since we can do basically whatever we want with rankings, we might as well use it to provide incentives to produce a "public good" that might be underproduced otherwise. And that public good is... legal scholarship.
He puts it this way: "Rankings can be used optimally, from a social standpoint, by directing the energy that will inevitably go into competition for high rankings for the creation of a socially desirable benefit that would otherwise go unproduced. Scholarship fits this description and, as such, this essay proposes, should be promoted explicitly by rankings systems."
So we need to encourage schools and their faculty to focus more on scholarship. Really?
Before I consider this claim more, let me make my own priors, commitments, and whatever elses clear. I love doing scholarship. Research and writing have been the focus of my professional life since I began my career more than 15 years ago, and it would take a lot to change that. I love engaging with other people's work, and when others engage with mine. I love SSRN downloads (to experience the rush as it floats effortlessly into your briefcase, see here). I abandon my wife and kids on nights and weekends all the time to produce scholarship, just ask them. I'll pit my stacks of unread law review articles against yours any day of the week; bring it on.
Moreover, I agree that faculties should be ranked by scholarly excellence to provide incentives for broadening and deepening our understanding of law, and Brian Leiter has performed a huge public service by doing this.
But my question is: what should we be doing with the U.S. News survey that exists (no one seriously disputes) for prospective students and employers? And my answer, contra Korobkin, is: create a race to the top on education, not scholarship, number of books in the library, or anything else.
Korobkin considers the possibility of ranking by the quality of education, and rejects it. His argument is: the education at one law school versus another is indistinguishable. And even if it was distinguishable in theory, we in the legal academy are not capable of distinguishing as a practical matter. Therefore, a race to the top would produce little marginal social benefit.
I'm no expert, but I think the research is at odds with both parts of this claim. Though I agree that at the moment no one has done the necessary work to aggregate the data and make comparisons among schools, thereby making it possible for busy deans and faculty to rank on educational quality, that doesn't mean it can't be done. It can.
Korobkin, though, thinks we should provide additional incentives for scholarship. But there is already a pretty strong incentive for faculty at most schools to focus as much of their time on scholarship, and as little as possible on education: the labor market. Which is all about scholarship -- that's how law professors get lateral mobility, raises, the good life.
Perhaps Korobkin means that without such incentives, schools will focus too much on hiring promising educators, as opposed to promising scholars. My sense is that we're in no real danger of that any time soon -- if that happens, let's talk in 10 or 20 years and revisit. But maybe I'm wrong -- what do you think of Korobkin's argument?
If you agree with me that the leading arguments out there -- Korobkin's and Leiter's -- are less than compelling for doing the U.S. News survey based on faculty scholarship, then we're back to trying to assess the quality of the educational "program" by school -- which I'll start to tackle in posts next week.
TrackBack URL for this entry:
Listed below are links to weblogs that reference Do We Need to Further Incentivize Scholarship, not Education?:
As a law student, I completely agree. Where I'm working this summer, I've heard the same comments from a variety of attorneys: there's a lot they "don't teach you in law school," not regarding doctrinal issues, but regarding, for example, the soft judgment calls. I'm sure to an extent legal scholarship can proxy legal education, but unless it's an exact match there is room for improvement in pure educational terms.
I wonder if Leiter's point isn't the product of some self-selection. The students at higher-ranked schools go into academia and lead the perception of law school rankings. I'm not simply saying that legal academics value their work more highly (I'm sure that's part of it): I'm also saying that there is a large contingent of highly capable lawyers from other law schools who may be benefiting from some idiosyncracy of their particular law school's pedagogy, and that we may never know what this benefit is so long as we are judging legal success by clerkships and academic appointments.
I suppose my judgment is colored by the fact that I'm working with intensely capable people, many of whom are not from top-tier law schools, and I am reticent to simply chalk it up to "oh, well, they are outliers." It's almost demeaning to their education, but of course that doesn't make it false.
Posted by: AndyK | Jul 10, 2008 2:25:23 PM
We share the same values. But I don't think the thought experiment on the appropriate criteria to apply to the USNWR survey instrument is going to produce a useful payoff. Last time I checked, the USNWR academic scores were treading down over time. Why? Probably because respondents were filling them out strategically.
Regarding educational quality, if there really is a difference between schools, a more fruitful is to ask why legal employers are not more discriminating in their hiring processes--it is their money and their firm that is at stake. In our empirical work on rankings, Andy Morriss and I can isolate a rankings effect, a geography effect, a HBC effect, and a size effect (i.e., more students at Georgetown than Stanford, so it's worth the firms' time to interview). These variables explains about 75% of the variance in on-campus interviews. Yet, there is virtually no anecdotal evidence that law firms care about educational quality--and what little there is often goes to things like "better entering credentials than US News rank."
So here is the problem--the market, at least at the top end, is not rewarding law schools that invest in teaching and curriculum. So what's the payoff? NU Law and a few other schools are trying to "move the market." But right now, an honest assessment of the rankings suggests that employers do not differentiate among law school based on educational quality; it is all about inputs. In other words, as a law professor, the rank of your school determines the amount of "rent" you can extract from entering students. This explains why professors want to lateral to hiring ranks schools--more pay, more prestige, less teaching.
I don't enjoy these materialist conclusions. If the ABA were willing to provide data for large scale "value-added" studies, healthy competitive forces would create a market for educational quality. But right now, these markets have little information, and consumers (college students) are not as sophisticated as they could be--they actually trust law schools to tell them the truth. To my mind, the status quo is both inefficient and immoral. The ABA is completely asleep at the switch.
Thanks for writing up this entertaining series. Best regards, Bill H.
Posted by: Bill Henderson | Jul 10, 2008 10:24:15 PM
Bill, thanks for your thoughtful comment. I certainly agree with you that the status quo is both "inefficient and immoral," and that the problem is that there is little to no useable information about (and therefore no real market for) educational quality.
My question is whether -- even before we can get the data for the large-scale "value-added" studies to which you refer -- some more information about educational quality might help create a market that both law schools and employers will respond to. I'll take a first stab at how one might do this next week, and look forward to your thoughts and those of others.
Posted by: Jason Solomon | Jul 11, 2008 9:06:33 AM