« Rotations | Main | Garfieldlessness »
Friday, August 01, 2008
FSU Law Ranks Very Highly on the New Leiter Methodology
It may not please Jason in terms of what he's trying to measure in the race to the top, but according to Jim Rossi, our fearless number-crunching Associate Dean for Research, Florida State's College of Law compares quite favorably on the new Leiter Rankings methodology.
Jim reports: "we seem to fall right around [Leiter's] cut-off for the top 20, right behind Minnnesota (which has an average of 200), but a) this is based on numbers [Rossi] ran [yesterday] and b) [Rossi] really don't know how many other schools are in the same general ballpark since Minnesota is the lowest ranking school [Leiter] reports for the most recent citation numbers." Please let me know if Rossi is missing anything here. We'll be happy to correct it. And if you're a school who wants to run numbers similar to the setup below, then please feel free to share the stats in the comments as they appear in the table below with the date the stats were performed.
Name Total Cites Post-2000 Cites
in JLR in JLR
frederick /2 abbott 659 463
rob /2 atkinson 413 205
barbara /2 banoff 117 80
donna /2 christie 100 65
robin /2 craig 219 139
joseph /2 dodge 473 356
dino /2 falaschetti 7 5
steven /2 gey 681 287
elwin /2 griffith 93 64
adam /2 hirsch 380 269
tahirih /2 lee 118 70
wayne /2 logan 311 187
david /2 markell 272 189
gregg /2 polsky 116 85
david f /2 powell 8 7
jim /2 rossi 552 343
j.b. /2 ruhl 943 558
mark /2 seidenfeld 690 411
nat /2 stern 173 77
fernando /2 teson 490 218
manuel /2 utset 143 87
donald /2 weidner 263 171
==============================
Average 328.23 197.09
Posted by Administrators on August 1, 2008 at 10:04 AM in Funky FSU | Permalink
TrackBack
TrackBack URL for this entry:
https://www.typepad.com/services/trackback/6a00d8341c6a7953ef00e553e47ac78834
Listed below are links to weblogs that reference FSU Law Ranks Very Highly on the New Leiter Methodology:
Comments
I think Leiter's rankings are great, and should be used in part to evaluate faculties, law school deans, and university presidents and provosts. Just don't think they should be much more than a minor factor in how people evaluate schools for the U.S. News rankings; U.S. News does identify scholarship as a factor in assessing academic quality of school's JD program but does not specify weight to be given. See my post on why it should not be given much here: http://prawfsblawg.blogs.com/prawfsblawg/2008/07/what-would-leit.html.
Posted by: Jason Solomon | Aug 3, 2008 1:30:55 PM
The only thing you're missing, which I missed too when I ran the numbers for GMU (which put us at 226, btw), is that Leiter is relying on the data from last Fall when he did the prior survey, not the new data. So you would need to cut off the citations in September or so of 2007, which would lower the numbers somewhat.
Posted by: David Bernstein | Aug 2, 2008 4:20:28 PM
Based on my calculations, the mean post-2000 citation count for tenured faculty at St. Thomas (MN) is 183. I would be interested in seeing a range of scores for schools outside the top 20.
Posted by: Rob Vischer | Aug 1, 2008 6:42:39 PM
Fair enough. This was originally intended for internal assessment purposes -- and not as a marketing claim of any sort -- and what Dan quotes from my original email to our dean and some others it is clearly tentative -- "we seem", we don't know know the data for other schools, etc. Until data for other schools are available we just don't klnow where FSU stands, but we are not that far behind where GW was based on data from a year ago (although they are presumably a bit ahead of us today).
Posted by: Jim Rossi | Aug 1, 2008 3:18:44 PM
I largely agree with your analysis. It's simply that it makes this claim a bit deceptive: "we seem to fall right around [Leiter's] cut-off for the top 20"
Posted by: merevaudevillian | Aug 1, 2008 1:32:36 PM
As to the "common name" concern, I think that the only common name at FSU for which this is an issue is David Powell. I used his middle initial, which is more accurate than Leiter's method. But in any event the citations at issue are so low (7, the second lowest on the FSU faculty) that any impact due to "overstating" is trivial. I did eyeball those I thought might be problematic by viewing the first 20 hits, but if I am missing any other common names please let Dan or me know.
But as to "understating," I also did not round to the nearest 10, as Leiter reports was done in his earlier study. I don't klnow whether this was done for each individual faculty member or at the end, but either way this would increase FSU's average numbers further.
It is a fair point that collecting data for one school does not allow any over- and under-inclusive errors to systematically apply across the the various schools being ranked. So Leiter's data, which is consistent across all schools in collection method, is superior to one school snapshots I or others might provide. (I see this as one of the chief public good provided by Leiter, U.S. News and other ranking efforts.) Of course, if someone to were systematically collect data below the top 20 schools Leiter reports, using a consistent methodology, this would solve the information deficit. But most of us have better things to do with our time.
Posted by: Jim Rossi | Aug 1, 2008 12:04:29 PM
As to the "common name" concern, I think that the only common name at FSU for which this is an issue is David Powell. I used his middle initial, which is more accurate than Leiter's method. But in any event the citations at issue are so low (7, the second lowest on the FSU faculty) that any impact due to "overstating" is trivial. I did eyeball those I thought might be problematic by viewing the first 20 hits, but if I am missing any other common names please let Dan or me know.
But as to "understating," I also did not round to the nearest 10, as Leiter reports was done in his earlier study. I don't klnow whether this was done for each individual faculty member or at the end, but either way this would increase FSU's average numbers further.
It is a fair point that collecting data for one school does not allow any over- and under-inclusive errors to systematically apply across the the various schools being ranked. So Leiter's data, which is consistent across all schools in collection method, is superior to one school snapshots I or others might provide. (I see this as one of the chief public good provided by Leiter, U.S. News and other ranking efforts.) Of course, if someone to were systematically collect data below the top 20 schools Leiter reports, using a consistent methodology, this would solve the information deficit. But most of us have better things to do with our time.
Posted by: Jim Rossi | Aug 1, 2008 12:02:23 PM
If I recall correctly, I wonder if FSU is overstating its results, per Leiter's test for false positives (which, at least from your post, doesn't include such a test):
"To guard against false positives with common names, ten to twenty of the 'hits' were reviewed; the percentage that were false positives was then multiplied against the total number of hits returned, and that amount was subtracted from the citation total."
Posted by: merevaudevillian | Aug 1, 2008 10:27:01 AM
The comments to this entry are closed.