« Socio-Economic Rights III (and mandatory ADR) | Main | Prawfsfest! in the Hassee »

Wednesday, April 29, 2009

A Law School Oral (Argument) Exam: Should an Oral Component be Added to the Law School Examination Process?

This will be my last post here at PrawfsBlawg, and I again want to thank Dan Markel for having me as a guest blogger for the month of April. I also want to thank the PrawfsBlawg readers for your helpful comments on my posts. 

In the wake of the Carnegie Report, law schools and law professors across the country are funamentally re-evaluating the way that legal education is being conducted. Recently I read a great article, From Grimm to Glory: Simulated Oral Argument as a Component of Legal Education's Signature Pedagogy, 84 Ind. L.J. 589 (2009) by Lisa T. McElroy. In the article, McElory "propos[es] that law professors regularly use simulated oral argument exercises to supplement traditional Socratic dialogue, [thus] meet[ing] head on the concerns expressed by Best Practices and Carnegie that over-reliance on the Langdell method neither mimics law practice nor nurtures student learning."

The article got me thinking that, at least in one way, law school is an enigma wrapped in a paradox. Throughout the semester, law students, more so than maybe any other students, learn material through their oral participation in class, under some version of the Socratic method. At the same time, law students (at least in substantive classes) complete written work for evaluation throughout the semester less so than maybe any other students. And yet, when the end of the semester rolls around, these same law students are tested almost exclusively based upon the quality of their writing and almost never based upon the quality of their oration (besides minor bumps up or down for class participation).

Of course, it wasn't always that way. As Ron M. Aizen notes in Four Ways to Better 1L Assessments, 54 Duke L.J. 765 (2004):

Today, first-year law students typically receive course grades based entirely, or almost entirely, on single end-of-course essay exams. Using a single exam to measure law student performance contrasts markedly with earlier practices at American law schools. From the early- to mid-nineteenth century, students were generally assessed far more frequently than they are today. For example, at the Litchfield Law School, the first professional American law school, students took weekly oral exams. Harvard examined students orally or in writing both weekly and "at the end of each text or topic." Michigan hired recent graduates and young lawyers to oversee daily oral and written examinations. Cornell, Penn, and Columbia combined frequent, often daily, quizzes with more cumulative assessments, such as end-of-term, annual, and graduation exams.

In the late 1800s, however, law schools began to use final exams as the sole measure of student performance. At Harvard in the early 1870s, students were for the first time required to pass annual exams to receive their law degrees. This innovation was introduced by Dean Christopher Langdell, who also popularized the case study method. The case method and the sole final exam allowed law professors to teach and evaluate large classes of students--and high student-faculty ratios were financially advantageous. Dean Langdell's economical model was eagerly embraced by other American universities. By the end of the nineteenth century, the use of single exams to assess student performance had become widespread among American law schools.

In the wake of the Carnegie Report, many have proposed and/or implemented some form of the pre-Langdellian practice of students completing written and oral assignments throughout the semester. I haven't, however, seen anyone proposing some form of oral examination at the end of the semester to match the oral learning that takes place throughout the semester. 

One reason would seem to be that most law professors would have no idea how to give an oral exam because we don't have formal education degrees and we weren't given oral exams in law school. A second reason would seem to be that it is not possible to give oral exams in classes subject to blind grading (i.e., the vast majority of law school classes).

But assuming that a professor could get a pass on blind grading in a class or that the professor were teaching a smaller class not subject to blind grading, what might an oral examination look like? My initial thought was to transform Professor McElroy's simulated oral argument technique into an exam. A professor could prepare something like a moot court problem with materials, have students prepare a brief, and have them participate in oral arguments. Such an oral exam would actually be an oral/written exam and allow students to be evaluated based upon both the quality of their writing and oration.

Do any readers have any thoughts about how a law school oral exam might look and whether it makes sense to add an oral component to the law school grading process? In doing research, the only discussion I found on the subject was in Oral Examinations as a Method of Evaluating Law Students, 51 J. Legal Educ. 130 (2001), where John M. Burman describes how he observed oral examinations of law students in Russia (which seem a lot like high school extemporaneous speaking competitions) and then applied a similar technique at the University of Wyoming College of Law:
Russia

Oral exams are scheduled for a time and date....All the students who wish to take the exam arrive at the appointed time, most dressed as for a court appearance. The testing procedure then goes something like this. Five students enter the classroom while the others wait in the hall. Each chooses one question from about fifty; the questions are typed on separate slips of paper and placed face down on the desk. Each student is then allowed time to think about the answer (without books or notes). As soon as one of the five is ready, she meets with the teacher. After she answers the written question, along with any followup questions, the teacher assigns her a grade, which is then entered into the student's grade book and on the official grade sheet that is later given to the administration. The student then leaves the room. As one student leaves, another enters and selects a question, and the process repeats itself, for hours, until each student has met with the teacher. 

Wyoming

I scheduled the oral exams for the second-to-last week of class. Instead of the cattle-call approach used in Russia, I took a more organized approach. I prepared and distributed a set of general instructions for the exams, which we then discussed in class.

I instructed each student to sign up to meet with me for twelve minutes. Thirty minutes before the appointed time, the student could pick up the exam from my secretary. The exam was similar in scope and content to written exams. It contained a fact pattern followed by three questions. The exam was open-book. Students could consult whatever sources they wished to outline or otherwise prepare an answer to the written questions. They were also instructed to expect additional questions from me.

I began each session by telling the student to begin wherever he wished. Most began with the first question, but others elected to start with the second or third. I found that I invariably began asking clarifying questions within a couple of minutes. Such questions are helpful in determining the depth of a student's knowledge. And an Are you sure? question yields interesting results. On the one hand, it may cause a student to reflect, perhaps take another approach, and demonstrate his knowledge. On the other hand, it allows students who are not well informed or prepared to confirm their lack of knowledge. To virtually eliminate the possibility of cheating, I use several different exams. My secretary rotates them randomly so a student does not know which exam she will have until it is time to pick it up.

I took copious notes during each exam so I could review my impressions later, as well as explain to students where they had done well or poorly. At the end of each day's exams, I classified the exams from best to worst and assigned tentative grades. At the conclusion of all exams, I reviewed my notes and the tentative grades before assigning final grades. Students were then able to get their grades from my secretary before finals, and several met with me to discuss where they had gone astray.

Posted by Evidence ProfBlogger on April 29, 2009 at 09:56 AM in Teaching Law | Permalink

TrackBack

TrackBack URL for this entry:
https://www.typepad.com/services/trackback/6a00d8341c6a7953ef01157053fa30970b

Listed below are links to weblogs that reference A Law School Oral (Argument) Exam: Should an Oral Component be Added to the Law School Examination Process?:

Comments

Colin - I've experimented. The first year it was 10%, 5% each for the oral argument and opinion. The second year I did it was 25%, 12.5% each. I've settled on 20% total, 10% each, now in the third year and think I like that the most. I count class participation another 10%, so the exam is 70%.

Posted by: David S. Cohen | May 1, 2009 12:32:32 AM

Thanks, mj. I imagine that it would scare certain students away and attract others, just as "paper" classes and closed book exam classes attract and repel certain types of students (as do "exam" and open book classes). Overall, though, I imagine that the type(s) of students it would attract would lead to very lively class discussions.

Posted by: Colin Miller | Apr 30, 2009 8:44:18 AM

Thanks, kathyrn. It seems as if European law schools rely much more on oral examinations than their U.S. counterparts. I wonder if there is some explanation for the difference.

Posted by: Colin Miller | Apr 30, 2009 8:42:32 AM

Thanks, David. What percentage of a student's grade is based upon these oral arguments?

Posted by: Colin Miller | Apr 30, 2009 8:41:12 AM

MJ:

I share your experience of departures at the margins, with larger classes. But I have been doing this project in smaller classes (15-18 students) with self-selected populations. Fed courts is going to be much larger next year (40+), so I am curious to see if it is different.

Posted by: Howard Wasserman | Apr 30, 2009 8:05:34 AM

Thanks for the helpful discussion. Glad to hear about the idea of a 20% grade for the oral assignment - it provides an incentive to work hard and spreads the risk. I'm intrigued, Howard, that you've found the skills uniform so far - my experience has been somewhat different. While there's a great deal of overlap, as you'd expect, there's been a surprising (initially) degree of difference - many tongue-tied students are the real analytical stars (in writing), and a number of gifted orators are quite unable to write at all.
You've been focusing on evaluation, Colin. This semester, I expanded the negotiation exercise of my family law class - the students seem to agree that it provided a great experience, but I have mixed feelings re moving it up to a clear % of the grade next time - does anyone think that scares students away? Draws them? Perhaps there's no good point of comparison among schools here?

Posted by: mj | Apr 30, 2009 12:00:46 AM

I had a couple of oral exams at LSU's summer abroad program, mainly from the European lecturers who examine their own law students this way. From what I remember, each student got 15-20 minutes and was examined directly by the lecturer. In one case there was another Professor present, I assume to help with another grading opinion since the exams weren't taped and to help with any language-barrier between the non-native and the southern accents.

I quite liked the examination process, though some of my "memorize the outline" classmates didn't. There were some easy "short answer" type questions, but the majority of the time was spent exploring a given hypothet and defending whichever position the student took on the problem. It worked very well for the seminar (comparative human rights law) and the doctrinal (EU Constitution) classes.

Posted by: kathryn | Apr 29, 2009 2:52:15 PM

I haven't given a final exam based on oral argument for my constitutional law course, but I do have graded oral argument as a significant portion of the graded coursework. Every student does one oral argument as an advocate and another as a judge. As a judge, they have to ask questions and then write a short opinion on one of the issues in the problem. The oral argument and the opinion count equally and significantly toward the final course grade.

It's a lot more work for the students and for me (especially in the grading and feedback), but I firmly believe it's worth it and the students, for the most part, enjoy it.

Posted by: David S. Cohen | Apr 29, 2009 2:17:01 PM

For the most part, the best oralist also writes the best paper--or at least the top 3 are the same. Occasionally you get an unexpectedly great argument from an otherwise only-decent student who writes an only-decent paper.

I agree that this is most appropriate for upper-level classes, although mostly because of class size. I have found that, even among 2Ls and 3Ls, only the very best students fully integrate stuff from across the curriculum into their argument. At the very least, though, oral presentation forces them to fully integrate everything from across the *syllabus*. Thus, I don't think it would be out of the realm of possibility to have an oral argument in my civ pro class (were I teaching smaller sections).

Posted by: Howard Wasserman | Apr 29, 2009 2:06:43 PM

Thanks, C.E. Petit. I agree that oral exams would be much better suited to upper level classes than 1L classes for the reasons you mention. Actually, my Criminal Procedure class this semester was what prompted me to think about the possibility of an oral exam. Another reason that oral exams would be better suited to upper level classes is that many students would have no desire to take oral exams, and I don't know that law schools should force them to take them. If oral exams were only given in upper level elective classes and professors were up front about giving them, students would have the choice to avoid classes with oral exams.

Posted by: Colin Miller | Apr 29, 2009 12:55:20 PM

My main concern here is that we're largely talking about 1L classes (and other required classes) in the examples, but that this sort of integrative exercise is more useful when the students have a base of knowledge to integrate -- that is, for 2L and 3L classes, which so many believe to "bore students to death."

Here's an example: Assume for a moment that it's a 1L contracts class being taught in the fall, and that the students have not yet had Civil Procedure, which at this law school is a one-semester class taught in the spring. Consider, for a moment, what the first question out of the mouth of a judge is likely to be, if one is using the "mock-brief-and-oral-argument" model: "Counsel, what is that standard of review here?" Without having had Civ Pro, the students are going to flounder on this and unfairly change the course of the argument.

Further, tailoring the exam technique to the integrative base of the students also fits better with what is well known about learning languages and culture. As anyone with a degree in English or History or any language could tell you, exams become more open-ended and integrative as the curriculum gets more advanced... yet we don't do that in law school. A current news example suffices: Many commentators in the publishing industry are surprised by the disclosure that the Justice Department is examining the propriety of the Google Book Search settlement. Anyone who had even taken antitrust law shouldn't have been... because one of the major cases lines of cases studied includes attacks on ASCAP and BMI. Given that the Google Book Search settlement establishes an "ASCAP for books" (without a quasicompeting BMI), even my dog figured out that the proposed settlement merits at least an inquiry. This kind of thought is precisely what one should expect of 2Ls, and especially 3Ls... but it was not thought of, or at least adequately considered, by any of the actual counsel in the Google Book Search settlement (it's nowhere in the settlement papers). Perhaps the only way to avoid this kind of an "oops!" is to implement testing of integration in law school.

Or, in the radical view, we could go to pass/fail for the doctrinal courses and use the only integrative course in the 1L curriculum -- legal research and writing -- as the graded course to separate the best from the others...

I'm certainly in favor of giving students more feedback in every class, but I'm not certain that the oral-exam route is particularly useful for 1Ls without an adequate base of knowledge. I'm much more in favor of short-answer, completed-in-class written works with highly focused questions (e.g., "State the holding in [major case x]. Does [x] control on the following facts [y(1)...y(n)], and why or why not?"), which has the bonus of allowing blind grading if that's a major concern, and can probably be graded at least in part by a research assistant. Really, now: If the rest of the university faculty gets to abuse student assistants, why can't law professors? ;-)

Posted by: C.E. Petit | Apr 29, 2009 11:49:47 AM

Thanks, JA. I know that WIlliam & Mary does something similar as part of its Legal Skills program, and here at John Marshall, we have the same type of thing. I know that at Baltimore, they have done a partial combining of substantive classes with skills classes, but I'm not aware of any pure substantive class where something similar is done.

Posted by: Colin Miller | Apr 29, 2009 11:11:28 AM

Thanks, Howard. Out of curiosity, after getting information about which student had which exam code, have you compared student performance on the oral and written components, and if so, have you found that two performances to be correlated?

Posted by: Colin Miller | Apr 29, 2009 11:08:40 AM

Not sure is this sort of simulation is standard, but our 1L legal methods class includes an extended simulation of briefing and arguing a motion before a mock Federal court. It's ungraded, but there's no reason why a doctrinal class couldn't include the same sort of simulation dealing with the subject of the class, yet evaluate it for a grade.

I suppose the biggest challenge would be the burden on the Professor's time to watch all the arguments and grade all the briefs.

Posted by: JA | Apr 29, 2009 10:54:16 AM

I have been doing this for the past four years or so in my upper-level electives (Federal Courts and Civil Rights). We hold oral arguments before the Supreme Court, using decisions from the courts of appeals (some of which actually are pending on cert). Each student has to argue one case and sit as a justice on one case and they are graded on both--no writing required. This counts for 20 % of the final grade-small enough that I am not running afoul of the anonymity requirement (bulk of the grade is a paper/exam hybrid, blind-graded), but big enough that the students take it seriously. We now do this as a full-day event, with food provided and everyone watching all the arguments. Most of the students seem to enjoy it and I am hoping word-of-mouth spreads that this is "part" of Wasserman's classes.

A colleague of mine described something he did a few years ago (I believe in Fed Courts, as well): He held individual meetings with students and would pick an index card out of a stack containing some term or concept and he and the student then would have a discussion about the concept, either in the office or while walking around campus.

Posted by: Howard Wasserman | Apr 29, 2009 10:15:57 AM

Post a comment