« Argument in Shapiro v. McManus | Main | Epps on Feiner »

Wednesday, November 04, 2015

Other Data I Would Like on Bar Passage--and Some Questions Not Entirely Susceptible to Data Analysis

I have learned a lot from both the serious and the casual work of Michael Simkovic. This is not entirely a matter of normative agreement, or at least I hope it's not, since the "greatest new scholar ever"/"worst villain since Stalin" dichotomy that tends to follow in the wake of work of his posts does seem normatively driven and rather silly. And I think it's both fine, and correct, to say that failure to pass the bar exam on the first try is not the end of a potential lawyer's career. That said, I would like to know more than he provides in this post.

In particular, I would like to know first-time and subsequent bar passage rates by school. It is true, as Simkovic writes, that the failure to pass the driving test on the first time is not generally viewed as precluding an opportunity to take it a second or third time. (Although perhaps we should be stricter with driving tests, and in any event my intuition about driving tests seems different from my intuition about, say, failing the medical boards multiple times. Driving and practicing medicine are both privileges, and both are dangerous activities. But no one thinks in terms of a right to be a doctor, and while driving helps in all sorts of primary activities, medical school is a long hard slog for the specific purpose of engaging in the practice of medicine and "MD-advantage" jobs.)

But if a driving school has a record of sending off its students to fail the driving test on the first attempt, and particularly if it consistently has a worse record than competitor schools with a similar client base, surely that suggests that there are problems with that driving school--even, in some cases, that the school is taking undue advantage of its students. Moreover, the worse the driving school's record is, the more likely I am to suspect that subsequent passage has much to do with the driving student's own efforts to study for and retake the test, perhaps more than once, and that the driving school itself did not contribute significantly to that ultimate success. In such circumstances, it would be fair to conclude that if other schools achieve better results with something like the same cohort, students may be made worse off by having gone to the lower-passage driving school rather than another school. Turning to law and speaking more generally, perhaps the ability to pass the test on subsequent tries, possibly despite rather than because of the education provided by that particular school, indicates either problems with using the bar exam as a barrier to entry at all, or with insisting on an ABA-accredited law school degree as a prerequisite to taking the exam, or some combination of the two.      

I would also like to know more about subsequent careers. Simkovic focuses mostly on future earnings for those lawyers, asserting that first-time bar exam failure is probably correlated with lower earnings ability but arguing that this matters less than the value-added to those eventual lawyers of having attended law school. That may well be true--for the lawyers. But I'm also--perhaps primarily--interested in the clients, beside whose needs, problems, and disadvantages the current or future well-being and state of relative disadvantage of law students seems to pale in both magnitude and importance.

What is the disciplinary profile of lawyers who failed the bar exam the first time, or second or subsequent times, compared to that of lawyers who passed on the first round? This LSAC report on a study of lawyers admitted to the Connecticut bar between 1989 and 1992 found that "approximately 6.7% of the never disciplined group failed the Connecticut bar examination on at least one occasion prior to admission, as compared to a nonsignificant difference of 10.7% of the severely disciplined group (see Table 6), and 16.9% of the less severely disciplined group (p < .05, Table 5)." But the study has appropriate caveats about its limits and about problems obtaining some data, especially about discipline in other jurisdictions. And I would want to know more about whether lawyers taking the bar exam in Connecticut fit a similar profile with lawyers taking the exam in other states (like Florida or California) and attending a different array of schools. I also would want to know something about the number of complaints filed, not just discipline imposed. It's a problematic number, not only because complaints are not proof of wrongdoing but because some clients may be more likely to complain with or without reason. On the other hand, lots of grievances stemming from genuine problems of poor management or low competence or unfairness to clients are disposed of without proceeding to final disciplinary decisions.  

I don't know whether there are other studies out there. I do note this report based on an examination (by reporters, not professional social scientists) of a much larger number of people taking the qualifying exam for stockbrokers, which suggests that those who failed the test repeatedly had worse disciplinary records than those who passed the first time. I would similarly want to know, regarding the bar exam, whether the likelihood of disciplinary problems went up with multiple failures. After all, few people, if anyone, think that people have a right to take the bar exam an indefinite number of times. So we might be interested, for clients' sake, in setting a cap on the number of times one can take the bar exam. Fail me once, shame on you; fail me eight times, shame on us.  

 The story also notes that one response contemplated by the Financial Industry Regulatory Authority was to disclose to investors the prior failure or failures of the stockbroker. Perhaps we could balance the interest in allowing lawyers to take the bar exam multiple times against the need to protect clients and provide them with relevant information by requiring lawyers to disclose to clients their prior failure on the bar exam. Just as law students should be treated as adult agents and allowed to make their own decisions whether to attend law school, as long as the schools are providing accurate information, so clients might be given that information and permitted to decide for themselves.     

Most of these questions are just that--questions. I'm not criticizing what Simkovic does provide, or strongly disagreeing with the conclusions he offers; to the contrary, I'm grateful for the post. I simply would like to know more before signing on to particular conclusions or recommendations too whole-heartedly. I would especially want to know more about the welfare of clients as well as the future success of lawyers. And I take it Simkovic would agree that none of what he does say precludes a number of possible concerns and conclusions or normative views.

Taking it as a given that first-time failure does not say enough about subsequent success or competence as a lawyer to justify stringent rules barring subsequent retaking of the test, and that the first-time bar passage rate does not tell us everything about a law school that we would want to know, I take it one might still conclude, depending on additional data but also on some reasonable normative views, that: 1) a high first-time failure rate might indicate problems at that school; 2) high subsequent failure rates would also be cause for concern about that law school; 3) if first-time or, perhaps more likely, repeated failure of the bar exam correlates to a greater likelihood of subsequent complaints or discipline, that should be cause for greater concern, given that the primary concern of state bars and accrediting agencies, if not the law schools themselves, should be the welfare of clients; 4) other measures, like disclosure of failures to pass the bar exam, might help address concerns for client welfare and are not out of the question; 5) given the relevant information, we might be able to think productively about setting a cap on the number of times one can sit for the bar exam; and 7) none of this is dispositive on the question whether barriers to entry into the legal guild should depend on one or both of a degree from an ABA-accredited law school and/or passage of the bar exam as presently constituted. 

Finally, a few words on this passage:      

It would be strange if newspapers claimed that those who fail a road test on the first try are doomed to never obtain a drivers license, will never be able to hold down a job, and should never have enrolled in high school in the first place.  But in the world of legal education, members of the press too often make comparably misinformed claims about law students and the bar exam.

It's just an analogy, of course, and thus of limited probative value; lawyers over-rely on arguments by analogy, in any event. As it happens, I agree with some of what it suggests. But it's a little odd. The most serious concern with the analogy is that it links the driving test to education in general, not to driving schools in particular. We generally require every child to enroll in high school, treating it as a gateway to responsible work and citizenship in general; we do not treat a legal education as necessary for all. If a particular high school failed to graduate a substantial number of its children, we would of course want to know what external factors, such as poverty, contributed to that. But we would also worry that the school was failing in important respects, especially if similar schools with comparable cohorts were doing a better job of education, as reflected (imperfectly, perhaps) in graduation rates. It would be strange if someone claimed that someone who fails to pass a driving test on the first try is doomed never to be able to hold down any sort of job, driving-related or not, but I've never heard such a claim. Conversely, given appropriate data, one might reasonably be more concerned about whether those who fail a driving test two, three, or more times should be employed in a job whose primary activity is driving a fast, heavy, dangerous vehicle. Concern for the future earnings of that individual might be viewed as less important than the possibility of that individual inflicting harm on innocent third parties. Newspapers may well be wrong to say (if they have said it) that a first-time bar exam failure suggests that the individual will never be a competent practicing lawyer and should never be able to retake the test, and should not even have been able to enroll as an undergraduate. They might be on more solid ground if they suggested that schools that show a low bar passage rate as compared to similar other schools might be doing a disservice to their students and might have problems that need to be addressed. And they certainly ought to factor the well-being of clients, and not just lawyers and law students, into their reporting or opining. Again, however, it's merely an analogy and less important than the material provided in the rest of the post, for which I'm grateful.       


Posted by Paul Horwitz on November 4, 2015 at 10:35 AM in Paul Horwitz | Permalink


The comments to this entry are closed.