« Scalia in the Hassee | Main | Typo Fun »

Thursday, April 08, 2010

Existential Crisis: The Leiter Site

Is it just me, or do the scholarly citation rankings recently posted on Brian Leiter's blog cast others into despair?  Seriously, though, I'm taking the posting of the citation rankings as a  good opportunity to remind myself to focus on the aspects of my scholarship I can control.  I can recommit myself to do thorough, systematic research. I can devote serious thought to topics that fascinate me and try to make my writing reflect that thought.  I can edit my articles to within an inch of their lives.  After that, citations will either come or not as fate dictates.  At the end of the day, I'll still have one of the best jobs in the world, which should be enough for anyone.

Posted by Lyrissa Lidsky on April 8, 2010 at 02:15 PM | Permalink


TrackBack URL for this entry:

Listed below are links to weblogs that reference Existential Crisis: The Leiter Site:


And don't worry, professors. This obsession about citations is only one of the reasons we students look upon you as rampant narcissists.

Posted by: Anonsters | Apr 11, 2010 3:40:26 AM

Law professors are very competitive people who like to rank themselves. There are lots of grounds to do that ranking: rank of where you teach, positions (such as chairs). prestige of journal placement, number of citations, etc.

In my view, it's all good so long as the rankings measure something that has a plausible claim to reflecting some kind of *real* thing. And citations fit the bill: They're a very imperfect reflection of impact, but they are at least *something.*

Posted by: anonanon | Apr 9, 2010 3:48:36 PM

If you are Professor Susan Smith with 300 citations, get married and become Professor Susan Jones or become Professor Susan Smith-Jones it seems like you lose your history of cites, unless you're diligent enough to check yourself and share that information with Professor Leiter (who graciously re-runs the numbers).

I imagine its pretty rare for a woman with a serious history of publishing to change her name when she gets married - the few exceptions can probably contact Leiter directly and, anyway, frankly the reason it's rare is that women know exactly what trade-off they'd be making when they get married and change their names despite a significant record of accomplishments under their maiden name.

Posted by: Katie | Apr 9, 2010 8:10:31 AM

Lyrissa, for us women the existential crisis runs deeper than that. if [just hypothetically ;) ] one happens to be the only woman on her institution's top 10 list, one might feel some obligation to stay on that list. i'm just saying...

Posted by: Orly Lobel | Apr 9, 2010 2:06:48 AM

I'm used to the process of trying to amass data in other areas - legal and non-legal aspects of corporate activity and law firm work, for example - in which we try to measure the results that tell us whether someone's performance is good, bad, or indifferent. Even when you think it would be easy to measure the results, it can be more complex than it seems. In executive compensation, for example, it ought to be clear that we measure tangible results. But did you really cause the stock price to go up or were all the boats rising on a rising sea? Or did you perform admirably only to have factors you can't control affect the results?

The valuable thing about citations is that they are indeed a tangible result in an area where answering the question "who is good at what they do?" is very, very difficult. For example, are lots of "string cite" references more valuable than a single citation in which another scholar engages with your work (assuming you don't have both)? Wise evaluators always look at any particular metric as simply one dial on the dashboard, and try to ameliorate, when setting evaluative metrics and counseling those being evaluated, all of the behavioral heuristics and biases. Put more simply, it's almost a law of organizational behavior that people will respond to what they are measured on, leading to what I used to call sub-optimization of results. Tell somebody that we're looking at inventory turns this quarter, and they might well bias the results to turn inventory, even if it means extending credit or credit terms such that you never get paid for all that inventory you are turning. (The sub-optimization comes from the fact that both inventory and accounts receivable are part of working capital, which you always want to reduce. Reducing inventory but increasing receivables defeats the whole purpose.) The example par excellence of this in the academy is the U.S. News ranking, in which people get bent wholly out of shape for ordinal movements (ooh, we went up three places or down three places) that are almost meaningless on the underlying bell curve used to put the rankings in order, and might well manipulate in real ways (fiddle with their part-time programs or stock up needless books in the library) to affect that meaningless result.

Personally, on the subject of citations, I discovered some time ago that Richard Posner had cited one of my articles in a 7th Circuit opinion for the wholly banal (and almost completely unrelated to the point of the article) definitions of insolvency in the equity and the bankruptcy sense. I have no idea why. Maybe a clerk did a Westlaw search and it was the first thing that came up? But I did let the deans know I had been cited by Judge Posner.

Posted by: Jeff Lipshaw | Apr 8, 2010 5:42:49 PM

The problem goes well beyond citations. I just posted a short piece that takes on the ranking/citation/download culture of legal scholarship and suggests that maybe there's a healthier way to look at all this, "Therapeutic Jurisprudence and the Practice of Legal Scholarship": http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1585311.

Here's the abstract:

The culture of legal scholarship has become preoccupied with article placement, citations, and download numbers, thus obscuring a deeper appreciation for the contributions of scholarly work. This article proposes that therapeutic jurisprudence (“TJ”), a theoretical framework that examines the therapeutic and anti-therapeutic properties of the law and legal practice, provides us with tools for understanding and changing that culture. More prescriptively, the article applies a TJ lens to: (1) identify a set of good practices for legal scholarship; (2) examine the TJ movement as an example of healthy scholarly practice; (3) consider the role of law professors as intellectual activists; and, (4) propose that law schools nurture a scholar-practitioner orientation in their students to help them become more engaged members of the legal profession.

Posted by: David Yamada | Apr 8, 2010 5:21:30 PM

I'm curious if anyone knows how the methodology accounts for female professors who get married and adopt their husband's last name?

If you are Professor Susan Smith with 300 citations, get married and become Professor Susan Jones or become Professor Susan Smith-Jones it seems like you lose your history of cites, unless you're diligent enough to check yourself and share that information with Professor Leiter (who graciously re-runs the numbers).

If I'm correct, we should encourage our colleagues who are concerned about their citations to self police as it's unrealistic to expect Professor Leiter to know of such name changes. Am I analyzing this correctly?

Posted by: Greg McNeal | Apr 8, 2010 4:34:20 PM

Lawrence, I agree. And by the way, I really respect your work. Anon, I also agree that citation counts aren't all they're cracked up to be. I have one article that has been cited far more than the others because I hit the timing just right and it came out just as defamation by anonymous speakers on the Internet was become a really hot topic. As far as quality goes, I think it is better than some of my articles and worse than others. But I don't know how to get the timing right every time, and I don't write quickly enough that I can just toss something off on every hot topic in my area. Incidentally, if I were really intent on gaming my citation counts along the lines you suggest, I guess I'd cite all of them by name right here and provide the link to my ssrn page. :)

Posted by: lyrissa | Apr 8, 2010 4:24:58 PM

I try to remind myself to focus on the intrinsic rewards of doing the best work that I can do. And in the end, I think aim at the intrinsic rewards is the best way to get the kind of recognition that really counts--serious engagement with your ideas!

Posted by: Lawrence B. Solum | Apr 8, 2010 4:06:24 PM

Or, if you don't want to have others know your citations, you might teach at a law school that is not considered significant enough for Leiter to study. I notice that Florida didn't make the cut, so no need to worry about how your cites compare to your colleagues and whether you made the top 10 on your faculty. If you teach at one of those schools studied you have it worse, as everyone one sees who is in the top 10 at each school.

Posted by: anon | Apr 8, 2010 4:01:15 PM

You actually *can* control your citation count, at least to some extent. The question is whether it makes sense to try to do so, which is a corollary of the question whether citation counts are a meaningful metric.

For example, you choose your subjects to write about based on their likelihood of being cited, and on this point, you can do studies of articles in your field that are often cited to determine which direction to go. You can publish a lot of articles, and each time include citations to every work you've written, however tangential. You can also watch SSRN for relevant works in progress and email authors suggesting that they cite you (I've been the recipient of such emails, though not the sender). You can also blog about your articles and provide citations, so that it is easier to find and cite your work. Finally you can choose your subjects based on what top law reviews are most likely to accept, which will increase the likelihood that you'll be cited for even the most banal of the points you make in an article ("The First Amendment protects freedom of speech." See Lidsky, Yale L.J.)

You can't control as easily your institutional affiliation, but that helps too.

Thinking about what it might take to inflate your citation count makes one wonder whether it would be worth doing.

Posted by: anon | Apr 8, 2010 2:42:53 PM

The comments to this entry are closed.