« Courts Law on JOTWELL | Main | External Critiques (or Challenging Priors) »

Wednesday, April 13, 2011

More on asking questions at workshops: internal versus external critiques

Adam Kolber posted last month about how to ask questions at workshops and conferences.  I found myself agreeing with each of his five points, especially the first one, admonishing questioners to focus directly on the speaker’s thesis.  In this post, I want to investigate this point a little more by developing a distinction between two kinds of inquiries.

There are, I think, two categories of thesis-focused questions:  internal and external critiques.  (Here, I’m leaving aside many other categories of questions that Adam rightly dismisses in his post as unhelpful, such as the question that seeks merely to ask the speaker to comment about the questioner’s own work or subject-matter area.) 

An internal critique questions the validity of the logical steps that animate the paper’s thesis, while an external critique questions the validity of the theoretical construct from which the paper is written.  For example, consider a hypothetical paper in which the author argues that change X in law is normatively appealing because it will incentivize people to do Y, and more Y will effect a Kaldor-Hicks increase in aggregate social welfare.  An internal critique would question whether it is right that change X would actually increase the incidence of Y, or whether greater incidence of Y really would represent an increase in social welfare.  An external critique, by contrast, would question the premises underlying traditional cost-benefit (welfarist) analysis, such as by questioning whether maximizing aggregate welfare is the appropriate criterion to guide policymaking decisions, or by questioning the coherence of the idea of measuring welfare at all.  (We can imagine a variant of the external critique that accepts the premises of welfarism on a higher level of generality but then argues that the Pareto criterion is the better one.)

Below the fold, I explore in more detail the challenges, costs, and benefits of each of these forms of critique.

First, both external and internal critiques strike me as validly thesis-focused questions because they seek to question the validity of a paper’s thesis.  An argument can fail either because its internal logic is flawed (internal critique) or because the major conceptual premises that animate it are flawed (external critique).  However different they may be, internal and external critiques share this quality: if they are right, then the thesis of the paper is wrong.

Internal critiques strike me as the favored, and by far more common, form of questioning a paper’s thesis.  Questioners typically focus on the coherence of the internal logical steps of the paper given the author’s worldview, and it’s hard to find much wrong with this.  External critiques, though, are less common, and in some venues, even unwelcome.  Their challenges are severalfold.  They are far from the core of the thesis, so the speaker may be less prepared to answer them, and so that they may take the discussion away from the paper itself.  Moreover, they may challenge someone’s basic worldview on such a fundamental level that responding forces the speaker into a general defense of their essential priors, and/or lead to an impasse to the extent that many of these priors are indefensible preferences about how the world works.  Finally, external critiques may be too effective if right.  While flaws in a paper’s internal logic may be fixable, what should a presenter do if they somehow become convinced that their entire worldview is wrong?  "Scrap the paper and write another one" may be the only answer.

I’m reminded of an externally critical question that was posed to me at a workshop.  My paper sought to examine a puzzle within property law.  A questioner who is a fan of informal systems of order pointed out that the puzzle tended to unravel if property law were defined broadly enough to include extralegal forms of regulating ownership.  This was a perfectly cogent question; it is true that my paper proceeded along a certain definition of what property law is, and it is also true that if that definition were changed, my paper would not work because the puzzle would no longer be the same.  My response, to the extent that I recall, was something like, “Yes, it’s true that my paper presumes a certain definition of property law, and I think that definition is valid for X and Y reason, though if it were otherwise then the paper wouldn’t work.”  The problem with this question was not that it was unfair or intellectually dishonest or not well-meant, it was that answering the question would have caused the dialogue to veer off in another direction, and so I was loath to elaborate in more detail about my answer to it. 

The reason I was disinclined to do this was that I wanted the discussion to focus (as the other questions had) on the validity of the paper's thesis, and because I didn't see the definitional question as useful since I was fairly well-satisfied with my take on that issue, and didn't see myself accepting a radically different one (and, in the process, scrapping the project entirely).  I suspect most presenters would share this reaction, at least to some extent.  If you have a civil recourse take on torts, or a retributivist take on criminal law, or a social-constructivist take on gender, then if someone merely says "if you were not a civil recourse theorist/retributivist/social-constructivist, then things would be different," is both so self-evidently true that it may not require discussion, and so far afield from the paper's topic that it would lead to the kind of debate that begins to be divorced from the thesis itself (and thereby violates the Kolber principle from the post that inspired this one--"thesis, thesis, thesis").

Having written all this, I’m finding it difficult to defend the poor old external critique.  This is concerning because as I wrote above, the external critique is as theoretically valid challenge to a paper’s thesis as an internal critique, so sidelining it completely in the name of expediency also seems like a non-ideal outcome.  Without external critiques, we all might become unduly complacent about our priors.  So I’ll conclude with a question:  Have I been unfairly critical of the external critique?  Is the internal critique really a fairer or more productive way to investigate the thesis of a paper?  Does the external critique have a place in workshop and conference Q&A, or should we do speakers the favor of accepting their priors even if we don’t agree with them?

Posted by Dave_Fagundes on April 13, 2011 at 09:37 AM in Life of Law Schools | Permalink

TrackBack

TrackBack URL for this entry:
https://www.typepad.com/services/trackback/6a00d8341c6a7953ef0147e42b207d970b

Listed below are links to weblogs that reference More on asking questions at workshops: internal versus external critiques:

Comments

I was going to comment on this post, but I couldn't decide whether to offer an internal critique or an external critique ("Why do you care about conference workshops anyway?").

Posted by: Stuart Buck | Apr 14, 2011 5:28:57 PM

Good stuff, Dave. My view tracks yours: internal critiques are almost always more useful to the thesis-presenter, whereas external critiques are usually less useful to the presenter, but often necessary as a part of larger discussion about appropriate paradigms or priors.

For that reason, in workshop settings, I virtually always attempt to offer (or solicit) internal critiques. As a workshop attendee, I might mention an external critique -- if I think there's one likely to be on the forefront of the minds of some imagined audience for the piece -- but often my goal there is to suggest to the presenter that (s)he appropriately "flag" that issue and specify that his/her interest is not debating first principles, but debating consequences once certain priors are taken as given. That's why, at first pass, I think external critiques have some place in workshop settings: they force the author (which in some cases has been me) to be a little clearer about my assumptions. In some cases external critiques have resulted in whole other papers, in addition to the discussed paper.

External challenges, in my view, are most useful when the conference/panel is explicitly framed as an external debate, because everybody knows the score going in. Similarly, a paper written with the aim of making a big-picture external critique of another piece is of enormous interest to me when I am educating myself about or researching a particular topic.

Posted by: Brendan Maher | Apr 13, 2011 4:30:06 PM

Marc, you rightly point out that my internal/external distinction applies best to the traditional kind of paper that seeks to advocate a change in law or that makes a descriptive critique, each from an established normative point of view (e.g., the hypothetical example in my initial post).

So, yes, not all papers do this, and for papers that don't have this basic form, external critiques may be much more appropriate. Consider Pierre Schlag's work from early in the 1990s, when he offered an influential critique of normativity in two articles from the Stanford and Texas law reviews. Since Schlag's work was explicitly about the coherence of normative scholarship, it's hard to imagine a question that wouldn't go to his priors and investigate the nature of scholarship itself. In such a context, the difference between internal and external critique may collapse, because they're one and the same. If you're considering Schlag's critique of normativity, then questions about the plausibility of priors _are_ questions about the internal coherence of his claims. (Ditto some of his more recent work, such as one of my faves from (I think) 2006 in the Georgetown Law Review, "Air Law, or the Rank Anxiety of Nothing Happening").

By contrast, if the paper we're talking about is more akin to my hypothetical traditional welfarist one above, then asking about the coherence of making normative judgments takes on a very different cast. It's a question that validly challenges the claim of the paper, but does so at a high enough level of generality that it's hard to think of it as a fair question, in the sense of being the kind of thing we expect a presenter to have carefully thought about.

Posted by: Dave | Apr 13, 2011 2:14:12 PM

Dave, nice post, and some questions to meet your question. How do you feel about scholarship which is itself an external rather than an internal critique of a certain mode of academic engagement? If one writes pieces which are themselves designed to be external critiques, is an external critique of that sort of piece of scholarship in turn more legitimate? Or do you still think that it's better to approach that sort of paper from an internal perspective?

Posted by: Marc DeGirolami | Apr 13, 2011 9:51:57 AM

Post a comment