2016 Meta-Ranking of Flagship US Law Reviews

This post was written by Bryce Clayton Newell of Tilburg University.

I want to thank Howard for posting this to the blog for me. I have been an avid reader of the blog for a number of years now, and it is nice to have the chance to try something out with you all and get some feedback on an idea for a new way to rank law reviews.

(I realize I may have just scared some of you off :) If not, I appreciate your continued attention.)

I am a long-time Angsting Thread lurker and sometimes commenter/spreadsheet contributor (when I have something to add). This submission cycle, during my “free” time between repeatedly hitting “refresh” on the Spring 2016 Angsting Thread (to read updated comments) while waiting for two articles to get accepted, I put together a meta-ranking of general/flagship US law journals. Law journal rankings show up periodically on PrawfsBlawg (as well as on other popular law blogs), but the semi-annual Angsting Thread continuously includes comments and questions about how to compare offers, whether to use US News rankings (either the Overall Ranking or the Peer Reputation ranking), and how (or whether) to consider alternative, impact-based, metrics like the W&L Combined Ranking or Google Scholar Metrics.

As a junior academic and aspiring prof., I’ve also asked these questions of numerous mentors and former law professors of mine. The advice I’ve gotten generally mirrors the advice I’ve seen in the comments: generally use the US News Overall Ranking (or maybe the peer reputation ranking, although this is less frequently mentioned), and if you can find it, consider the average ranking of a school over the last X years rather than only the most recent annual ranking; the alternative rankings can be useful to e.g., compare a specialized journal with a flagship one or as a way to distinguish between offers from two closely ranked (by US News) general journals, but they should not replace the general consensus that (one of ) the US News rankings is the best gauge of journal prestige.

However, as an interdisciplinary scholar who publishes work in both peer-reviewed social science journals and law reviews, it strikes me as odd that we would discount measures of journal impact completely when choosing where to submit and which offers to prioritize. To be sure, the W&L ranking has some flaws (some described earlier on PrawfsBlawg here) and other citation-based impact factor rankings commonly used in other disciplines (like JCR/ISI) also have their limitations (including poor coverage of law journals). Google Scholar Metrics represents an interesting alternative way to measure impact (Google’s metrics description is here), but also doesn’t have full coverage of law journals and comes with a different set of concerns. Regardless, though, it appears important that some measures of citation or impact are taken into account, as direct correlations between US News rankings of law schools and law journal importance seem a bit weak as the primary (or only) measure to evaluate.

So, to get to the point, I decided to create a meta-ranking of the possible contenders for gauging the relative importance of journals and offers: US News Overall Ranking (averaged from 2010-2017), US News Peer Reputation Ranking (also averaged from 2010-2017), W&L Combined Ranking (at default weighting; 2007-2014), and Google Scholar Metrics law journal rankings (averaging the h-index and h-median of each journal, as proposed here by Robert Anderson). I've ranked each journal within each ranking system, averaged these four ranks using a 25% weighting for each, and computed and ranked the final scores. I think this approach benefits from incorporating a couple different forms of impact evaluation (W&L + Google) while not disregarding the general sentiment that law school “prestige” (USN combined rank + peer reputation rank, each averaged over an 8-year period) is an important factor in law review placement decisions.

I would love to get feedback about whether you think there is any usefulness to doing this in this way, whether you would suggest alternative weightings, different combinations of rankings, or if I have overlooked something (entirely possible, as I was paying more attention to your comments on the Angsting Thread than anything else when I put this together), etc. If it seems that folks are interested and that this might be useful, I can also post full ranking (I’ve ranked 194 journals). I am also working on an attempt to evaluate equivalencies between specialty journals and general ones, and I’m happy to take suggestions or share my initial thoughts on doing that if you’d like to get in touch.

The big movers here (in this ranking versus the average US News Overall Rank from 2010-2017) seem to be (but there are quite a few others who moved around):

• New York Law School moved up a whopping 38 places (to #99);
• Vermont moved up 31 places (to #91);
• UC Irvine dropped 30 places (to #59);
• Akron moved up 28 places (to #99);
• Albany moved up 27 places (to #96).

Journals like Fordham (#26, up 10 places), Hastings (#36, up 12 places), Cardozo (#42, up 18 places), American (#46, up 11 places), and Lewis and Clark (#53, up 23 places) that have been frequently referred to in Angsting Thread comments as “hitting above their weight” all also improved at least 10 places (as did Missouri, Connecticut, Denver, Brooklyn, Chicago-Kent, Seattle, Oregon, Buffalo, Santa Clara, Indy, DePaul, South Carolina, St. Louis, Hofstra, Marquette, and Howard). Other journals dropping 10 or more places include: Arkansas-Fay., Kentucky, Georgia State, Temple, SMU, Arizona State, Georgia, and Alabama.

Other sizable moves in the top 20:

• Chicago (#12) drops 7 places (Google’s ranking moderated the even more drastic difference between Chicago’s US News rank and W&L rank);
• Iowa moved up 5 places (to #20);
• Northwestern dropped 4 places (to #16);
• Michigan (#6), Georgetown (#10), Texas (#11), and Notre Dame (#19) all moved up 4 places.

Finally, here are the top 100 ranking journals in the 2016 Meta-Ranking:

MetaRank	Journal	Change from USN Rank	MetaScore	Avg. USN Peer Rank	Avg. USN Overall Rank	W&L Rank	Google Rank
1	Harvard Law Review	1	1.5	1	2	2	1
2	The Yale Law Journal	-1	1.75	1	1	3	2
3	Stanford Law Review	0	2.75	3	3	1	4
4	Columbia Law Review	0	3.75	4	4	4	3
5	University of Pennsylvania Law Review	2	6.5	9	7	5	5
6	Michigan Law Review	4	8	8	10	8	6
7	California Law Review	1	9	7	8	12	9
8	New York University Law Review	-2	9.25	6	6	14	11
8	Virginia Law Review	1	9.25	9	9	9	10
10	The Georgetown Law Journal	4	9.75	13	14	6	6
11	Texas Law Review	4	12	15	15	10	8
12	University of Chicago L. Rev.	-7	12.75	5	5	25	16
12	Duke Law Journal	-1	12.75	11	11	16	13
14	Cornell Law Review	-1	13.25	12	13	15	13
15	UCLA Law Review	1	13.5	16	16	7	15
16	Northwestern University Law Review	-4	15.25	14	12	13	22
17	Minnesota Law Review	3	15.75	20	20	11	12
18	Vanderbilt Law Review	-1	17.5	17	17	20	16
19	Notre Dame Law Review	4	21.75	27	23	19	18
20	Iowa Law Review	5	22.5	27	25	18	20
21	Boston University Law Review	3	24.25	25	24	22	26
22	William and Mary Law Review	8	25.5	32	30	21	19
23	The George Washington L. Rev.	-2	26	23	21	29	31
23	North Carolina Law Review	11	26	21	34	28	21
25	Southern California Law Review	-7	26.5	19	18	32	37
26	Boston College Law Review	5	27.25	29	31	23	26
26	Fordham Law Review	10	27.25	35	36	16	22
26	Indiana Law Journal	0	27.25	30	26	27	26
26	Washington University Law Review	-7	27.25	18	19	37	35
30	Emory Law Journal	-8	27.5	22	22	36	30
31	Wisconsin Law Review	4	30.25	24	35	40	22
32	University of Illinois Law Rev.	6	31.25	34	38	24	29
33	U.C. Davis Law Review	-1	33	26	32	31	43
34	Florida Law Review	16	36	38	50	34	22
34	Washington Law Review	-6	36	37	28	30	49
36	Hastings Law Journal	12	37.25	36	48	33	32
37	Ohio State Law Journal	3	39	31	40	42	43
37	Washington and Lee Law Review	0	39	33	37	39	47
39	Arizona Law Review	4	39.25	41	43	38	35
40	Alabama Law Review	-13	40.75	42	27	45	49
41	Wake Forest Law Review	0	41.5	44	41	43	38
42	Cardozo Law Review	18	43	53	60	26	33
43	Georgia Law Review	-10	43.75	40	33	47	55
44	Connecticut Law Review	12	45.75	52	56	35	40
45	Colorado Law Review	0	46.25	43	45	50	47
46	American University Law Review	11	47	48	57	43	40
47	George Mason Law Review	-3	48.5	55	44	46	49
48	Brigham Young University Law Review	-6	49.5	50	42	54	52
49	Maryland Law Review	-2	50	47	47	61	45
50	Tulane Law Review	1	52.25	45	51	49	64
51	Utah Law Review	-5	56.5	51	46	57	72
52	Florida State University Law Review	0	56.75	49	52	58	68
53	Houston Law Review	1	57.25	66	54	51	58
53	Lewis & Clark Law Review	23	57.25	79	76	41	33
55	Pepperdine Law Review	-2	58.5	70	53	59	52
56	Arizona State L. Journal	-18	62.5	46	38	73	93
56	Loyola of Los Angeles Law Review	9	62.5	69	65	70	46
58	Missouri Law Review	22	65	65	80	60	55
59*	UC Irvine Law Review	-30	65.5	38	29	111	84
60	University of Cincinnati Law Review	7	66	74	67	55	68
60	University of Miami Law Review	6	66	54	66	67	77
62	Denver University Law Review	11	68	63	73	72	64
63	Brooklyn Law Review	16	69	67	79	53	77
64	Chicago-Kent Law Review	10	69.5	68	74	78	58
65	Seton Hall Law Review	2	70.25	84	67	78	52
66	SMU Law Review	-17	70.5	64	49	74	95
67	Tennessee Law Review	-4	70.75	62	63	65	93
68	The University of Kansas Law Review	7	71.25	61	75	81	68
68	Seattle University Law Review	24	71.25	89	92	66	38
70	Case Western Reserve Law Review	-9	71.5	59	61	85	81
70***	Penn State Law Review	0	71.5	91	70	67	58
72	Oregon Law Review	15	72.25	56	87	74	72
73	University of Richmond Law Review	-9	72.5	82	64	67	77
73	San Diego Law Review	-4	72.5	57	69	78	86
75	Buffalo Law Review	16	73.75	94	91	52	58
76	Temple Law Review	-18	74.25	60	58	84	95
77	Loyola University Chicago Law Journal	1	77.5	77	78	74	81
78	Santa Clara Law Review	22	78	74	100	74	64
79	Georgia State University Law Review	-21	80	76	58	114	72
80	Indiana Law Review	10	81	73	90	89	72
81	DePaul Law Review	23	83	99	104	48	81
82	South Carolina Law Review	19	83.5	94	101	71	68
83***	Rutgers University Law Review	3	83.75	72	86	91	86
84	Nevada Law Journal	-7	84.5	93	77	82	86
85	Kentucky Law Journal	-23	86	71	62	93	118
86	Louisiana Law Review	-3	86.5	101	83	98	64
87**	University of Pittsburgh Law Review	-6	87.75	58	81	106
88	Villanova Law Review	-3	88.25	86	85	96	86
89	Saint Louis University Law Journal	10	89.25	99	99	64	95
90	Nebraska Law Review	-8	90.25	78	82	88	113
91	Vermont Law Review	31	91	108	122	94	40
92	Hofstra Law Review	13	91.75	96	105	62	104
93	Marquette Law Review	10	92	87	103	83	95
94	Michigan State Law Review	2	92.75	102	96	55	118
95	Howard Law Journal	23	93	98	118	101	55
96	Albany Law Review	27	98	124	123	87	58
97	Catholic University Law Review	-2	98.75	90	95	97	113
98	Arkansas Law Review	-14	99.25	97	84	139	77
99	Akron Law Review	28	101.25	143	127	63	72
99	New York Law School Law Review	38	101.25	118	137	92	58

I have also calculated the Spearman's Rank Correlation Coefficient for the MetaRank versus each of the four underlying rankings (for schools ranked in the top 100 in the table above) and have charted the correlation (for all journals ranked 1-150 in the MetaRank).

	USN	USNpr	Google	WLU
MetaRank	0.913	0.930	0.867	0.902

Notes about method: Google Scholar Metrics are currently based on the most recent Google index (updated to June 2015). Some journals that began after 2007 are effected by W&L’s ranking formula described here. Because the US News Ranking has changed how it reports 3^rd and 4^th Tier schools over the relevant period, I have done the following: for the 2010 and 2011 editions of the ranking, I gave all “Tier 3” schools a value of 115, and all “Tier 4” schools a 150; for 2012-2017, I assigned all unranked schools (those not ranked 1-149) to 150.

* As some journals were not included in the US News Rankings for all 8 years (e.g. UC Irvine was only in the 2016 and 2017 rankings), I have averaged the ranks over just the years they appear, rather than by 8.

** University of Pittsburgh Law Review is the only top 100 journal not included in Google Scholar, as such, I have used W&L’s rank twice to keep the impact portion of the ranking at 50%.

*** Rutgers recently consolidated two law journals into one and Penn State now also has two ranked law schools but only one flagship law review. For each of these schools, I have used the rank in each category (W&L/US News, etc.) that is the highest. Similarly (although not appearing in the top 100 reported here), Widener split and was ranked separately in 2017 by US News (each school taking a separate flagship journal). As such, I have used the separate 2017 ranks from US News but shared the single Widener school ranking reported in US News in its 2010-2016 editions of the rankings.

Last note: I cannot locate the 2015 US News Peer-Reputation rank for Loyola-New Orleans (it is missing from Paul Caron’s annual posting at TaxProf, and I don’t have access otherwise). If someone has that information (or even the underlying scores themselves) and could pass it along, I would appreciate it.