« Tacit Citation Cartel Between U.S. Law Reviews | Main | I wanna be a billionaire so freakin’ bad --Bruno Mars »
Wednesday, September 05, 2018
Joint reponse to comments on the cartel post
Many thanks for all who have responded to my earlier post
We are very familiar with citation practices of US scholars but the purpose of the article was to try and think about them - also going beyond intuitions. I will make 2 quick points in response.
First, it is a valid question to ask how to distinguish between citation cartels and epistemically-driven scientific communities. From a network-topological perspective, these two forms of clustering seem indistinguishable: both constitute a group of nodes (which may represent scholars, institutions, or journals) that are more highly connected to each other than to the rest of the scientific world. Our argument - and I will try to give the detailed argument (which appears in the paper) in a later post - is that the kind of closure we see in US law reviews (the quantitative analysis can be found in the paper) cannot be defended on epistemic grounds. For example, one could argue that the reason for that greater closure is that US law reviews focus on domestic law. However, U.S. law journals have been criticized over the past few years for being too theoretical and for not displaying the right balance between theory and doctrine (e.g. Harry T. Edwards, ‘Another Look at Professor Rodell's "Goodbye to Law Reviews’ (2014)). They have also become increasingly more interdisciplinary (e.g., George L. Priest, ‘The Growth of Interdisciplinary Research and the Industrial Structure of the Production of Legal Ideas: A Reply to Judge Edwards’). If this is the case, the work that is published in PR journals - in the 'law and genre' - should be relevant for the work published in US generalist journals. If U.S. scholars do not cite articles in PR journals they might be losing good ideas. I think this point was missing from the responses.
My second point is that we are also trying to make a general argument about the risks of relying on rankings as a way to judge quality of research. Part of our objective is to expose some of the problems with the current practice of lumping the 2 categories together. I will further reflect on that in future posts.
Posted by Oren Perez on September 5, 2018 at 10:35 AM | Permalink
Comments
Who, exactly, is complicit in the "current practice of lumping the 2 categories together" for the purposes of evaluating research?
Posted by: lectuer | Sep 5, 2018 5:35:30 PM
Thanks for the follow-up, Oren. When you say that the inward-focus of US law reviews "cannot be defended," I'm not sure if that means that, normatively, it is less than ideal, or that, descriptively, it cannot be explained by non-cartel behavior (which I assume to be a claim of intentional behavior out of some perceived in-group self-interest).
But I'll echo BDG's comments, similar to ones I made over e-mail, that this may simply reflect US law professors being in an environment that is different from what you are assuming it is. For example, a lot of U.S. lawprofs do research using available databases that don't include many peer-reviewed journals, such as Westlaw's JLR database. And in many areas of law there are very few (if any) peer-reviewed journals that regularly feature articles in that field. The result is that works in the field that are in peer-reviewed journals may be more difficult to find than works in student-edited journals. I agree that this is less than ideal, but if it's a problem sufficiently serious as to require solutions, I would think the more direct solutions would be something like making peer-reviewed journals more accessible to U.S. law professors (such as including them in Westlaw's JLR database).
Posted by: Orin Kerr | Sep 5, 2018 3:45:35 PM
I'm sorry, but U.S. legal academics, unlike equivalent scholars in the social science disciplines, don't use Web of Science (JCR) or any of its direct competitors for in-discipline journal research--they use Westlaw and Lexis (and possibly HeinOnline). And Westlaw and Lexis simply do not include the fulltext of very many peer reviewed journals.* That means legal authors are far more likely to discover student-edited pieces. (On that note, I have to wonder who is writing all of these apparently peer-reviewed articles. I suspect that a close analysis comparing authors in student-edited journals vs. peer-reviewed journals would be interesting. Somebody must be writing the articles in the 953 peer-reviewed journals [??!] you discovered, but I doubt they are U.S.-based legal academics in most cases.)
*I got tired of looking, so I only checked the first 10 peer-reviewed journals used for your analysis. But of those 10 journals, 4 are not available through Westlaw (American Law and Economics Review, Behavioral Sciences & the Law, Chinese Journal of International Law, Common Market Law Review, and European Constitutional Law Review). By comparison, all of the first 10 student-edited journals are on Westlaw. I didn't check Lexis. (Even for journals that are on WL/Lex, coverage may not go back as far as it does for student edited journals.)
Posted by: AnonymousLawLibrarian | Sep 5, 2018 3:43:58 PM
I say this without having read the paper, so you may well say this in the paper, but if you want to find an example of an epistemic/citation community even more "closed" (actually MUCH more closed) than the one you've noticed in U.S. legal scholarship, you should check out the epistemic/citation practices of U.S. courts.
Posted by: Joey Fishkin | Sep 5, 2018 10:55:04 AM
The comments to this entry are closed.