« Should Sara Jane Olson Get to Leave California and Serve Parole in MN? | Main | My Spring Break... »
Monday, March 23, 2009
Multiple-Choice Law School Exams
I've informally asked around over the last few years about giving multiple-choice (or partially multiple-choice) exams and have noticed:
-
Views (and institutional norms) are strong
- Whether multiple choice is considered acceptable varies among subject areas (civ pro ok? con law no way?)
- Some of these strong positions are linked to issues about "teaching to the bar"
So, multiple choice - pro or con?
Posted by Verity Winship on March 23, 2009 at 01:41 PM in Teaching Law | Permalink
TrackBack
TrackBack URL for this entry:
https://www.typepad.com/services/trackback/6a00d8341c6a7953ef01156e440dab970c
Listed below are links to weblogs that reference Multiple-Choice Law School Exams:
Comments
I've nver heard a reasonable defense of multiple choice exams in law school. There use is mainly motivated by wanting to avoid grading. The problem that if you use them you are not fully performing the teaching job. Teaching means examining the reasoning and writing of students so you can identify problems. A pure tester only tests but does not examine result in order to know when specific types reasoning errors or writing problem need to be addressed.
Posted by: jeff | Aug 24, 2009 12:24:17 PM
My tax prof gave a 4 hour multiple choice exam. Approximately 1/3 of the 300 questions were cumulative. That is, to answer most of the questions, you needed to have correctly answered the first ones. This meant that a simple math error at the beginning could lead to failing the exam.
Despite studying like crazy and participating in class the whole semester, and really feeling like I knew the material, I got a B. This was bitterly disappointing.
The reason this exam was so hard, is that the answers were given in ranges $1,000 to $4000, $4,000 to 6,0000 etc. That meant that you couldn't tell if you were off track.
While I don't object to multiple choice questions where there is a definite best answer (like in Tax)it is important for professors to think about how different formats will affect different types of test takers.
I also think that multiple choice tests for "counseling" areas of the law, like classes involving family law topics, are ridiculous.
Posted by: Minxy | Apr 1, 2009 1:41:05 AM
I'm with Dave and Bruce. I use, in some classes, some multiple-choice questions, for some things (at least, I do sometimes). They do enable me (I think) to test more broadly (which is, I think, more fair to students) and, relatedly, to avoid the sillier extremes of "issue spotters" essay questions where all of (say) Criminal Law is crammed into one bizarro-day series of events. If an exam was *all* (or even mostly) MC, I'd worry that the professor (i.e., me) was just trying to avoid grading. But, that's not my reason, so I sleep alright.
I also, FWIW, avoid to the extent I can the traditional 3-4 hour death-frenzy-scribble (or type) exam format (opting instead for 8 hour sessions). Indeed, it seems to me that many of the criticisms levied against MC questions apply with (at least) equal force to 3 hour essay exams.
Posted by: Rick Garnett | Mar 25, 2009 5:10:50 PM
I wanted to mention that I think that multiple-choice exams are unlikely to be (or become) widespread in law school for a few practical reasons:
-As many of the comments have mentioned, good multiple-choice questions are difficult and time-consuming to draft. (And the work happens during the semester.)
-Although grading is quicker, a partially multiple-choice exam may be only marginally quicker. And many of the benefits of multiple-choice seem to come from mixing them with essays and short answers. (Maybe excepting Evidence, which Howard mentions above.)
At the same time, I agree with Dave that figuring out which legal issues actually are in a grey area seems like a worthwhile, lawyerly skill that multiple choice might highlight.
Posted by: Verity Winship | Mar 25, 2009 3:29:19 PM
I agree with some of the posters who believe that including a number of different types of testing works well. I usually include both m/c and written portions (either a series of short answers or a long essay) on my tests. For the m/c portions I allow the students to answer the question with an essay if they prefer. The language for this looks something like the paragraph below, but usually has some class specific component as well:
If you feel that the question is ambiguous or may lead to more than one answer, then you may answer it with a short essay. You may do this by: (a) clearly denoting the question that you are answering on the back of the examination, (b) stating the grounds for the ambiguity, and (c) fully answering the given question (a paragraph or more), including alternative answers for different ways that the ambiguous portions of the question may be interpreted.
Posted by: Jeff Yates | Mar 24, 2009 1:52:47 PM
I'm surprised there's so many multiple choice giving profs here; I think I only had one class in law school that used it.
A clarification: I don't think the objections to multiple choice is that there aren't right or wrong answers in law (at least, that wasn't my objection). It's that I don't want a focus on a quick right or wrong answers to be the last lesson my students take away from my class. I also don't trust myself to write unambiguous multiple choice questions (hell, as I recall from studying for the LSATs, even the test prep companies trying to replicate LSAT questions often couldn't manage to do it and it's their full time job). And I remember from being a student how utterly maddening it was to be presented with an ambiguous multiple choice question.
Posted by: anon | Mar 24, 2009 1:23:00 PM
When I first started teaching, I gave finals in Property and Bus Orgs that were 1/3 MC and 2/3 essay. I still have those proportions as far as final grade weighting goes, but I now give the MC portion as a series of graded quizzes throughout the semester (one at the end of each major unit). MC questions are very good for testing basic understanding of the law. The student response has been great -- students like getting feedback on their understanding of the material, and having to study for the closed-book quizzes puts them in a great position come finals time.
Posted by: Ben Barros | Mar 24, 2009 9:49:16 AM
There's an article in the recent J of Legal Educ that was written by people who are experts in psychometrics about the uses and disadvantages of m-c questions. It offers some good advice on how to craft good m-c questions and how to avoid writing (or using) bad ones.
Posted by: Dan Markel | Mar 23, 2009 9:18:30 PM
Wow. Opinions do indeed appear polarized. I use M/C in both copyright and property, though only for half the exam. It is not meant to be a racehorse part of the exam, but is roughly modeled on questions of the level of complexity that one might get on the bar. There are a few reasons I like mixing in M/C:
1. Legal issues are neither all black-and-white nor all shades of grey. While there are many hard issues, the law very often has clear answers that still take skill and sophistication to identify. Understanding this aspect of legal analysis is important (and often misunderstood, as the student's comment above indicates), and I want my exam to reflect this.
2. Writing well, particularly under pressure, is a very important skill. Nevertheless, I am skeptical that placing 100% of an exam's emphasis on writing accurately reflects all students' mastery of the material. Weaker writers who know the material very well may not reflect this on an exam (and the reverse--excellent writers may do better even though they do not know the law as well). M/C seems a good way to correct for this.
3. Breadth. With M/C, I can test all of the subjects we've covered in the class, while essays invariably cannot test nearly as many of them. In order to do well on my exam, you have to know all the aspects of the topics we cover, not just a few (related, the M/C exam rules out the possibility that students who have narrow or idiosyncratic knowledge of some sub-topic can still ace the exam).
4. Essay grading is always imperfect and to an extent subjective, no matter how hard we try to be fair. I think the biases and errors that profs make are fairly marginal, but having an objective portion of the exam makes sure that there is a corrective to the extent that they inevitably arise.
5. Bar preparation. This is a weak factor, but I think there is some value for the bar, at least in courses like property, both in terms of learning the material and becoming skilled at answering M/C questions.
6. Teaching benefits. Thinking up and drafting M/C questions has been a great challenge and has required me to resolve hard doctrinal issues, to separate issues with inevitable subjectivity from those that I think can be tested objectively, and to sharpen my in-class hypotheticals and problems (which are deceptively hard to do well). I don't always do this, but I often write up M/C questions immediately after class when new examples and problems are fresh in my mind (and which also assures that the M/C questions will be germane to class discussions).
Just my two cents. I know I'm very much in the minority on this.
Posted by: Dave | Mar 23, 2009 5:44:29 PM
What are we trying to do as law professors? That is the primary question, to my mind, when deciding whether to give multiple choice.
I think that Bruce is right. Multiple choice questions are good for some things, some of the time. The fact is that there ARE some right and wrong answers in the law. Many of them, in fact. And in legal practice, you do well do know that.
It is also worth noting that a good portion of the bar exam is multiple choice. I don't "teach to the test," but to the extent we can help prepare students in some ways for the bar exam, shouldn't we?
I give some multiple choice questions, some of the time, in some of my classes.
Posted by: Hillel Y. Levin | Mar 23, 2009 5:31:51 PM
I use it for part of my civ pro final. I find the strong views against above somewhat odd, and I'm curious what strongly held views Verity's come across. I'm not in love with multiple choice, but it has some advantages over essays: 1) There are certain discrete issues that are better tested by MC than by essay. There may be more of these in rule-based classes like civ pro or evidence. You can throw them into an essay question as minor points, but if they are missed, it's ambiguous why. 2) It is possible to test a broader range of subject matter, and thus be less prone to sampling error. 3) Law is murky, but there are some right and wrong answers. 4) Essays can go profoundly off-track with an initial misstep. How do you grade those? MCs are not subject to that problem. 5) Writing clarity is important, but I don't teach it, so I like to use a mix of measures when I can.
Posted by: Bruce Boyden | Mar 23, 2009 4:31:03 PM
I use exclusively multiple choice (or multiple-choice-with-explanation) for Evidence, for the precise reasons and with the precise objectives that anon 3:56 describes. And because I am trying to replicate a "trial," it works well. I have been forced by circumstances to go to a multiple-choice format for civ pro this semester--we will see how that goes.
Posted by: Howard Wasserman | Mar 23, 2009 4:10:08 PM
I had a multiple choice exam in law school. 240 questions (maybe more, I know it was at least one a minute) to complete in 4 hours. Believe me, when all was said and done, you knew whether you truly understood evidence and so did the professor. I finished in 1 hour and 45 minutes. It wasn't just memorizing and being able to explain my decision, taking the exam was like being in trial. I had to know the rules and how they worked and I needed to be ready to jump in with my objection and my reasoning with split second timing. No time to look it up while my opponent continued questioning and the timeliness of my objection was lost forever. That exam was the best preparation I had for the MBE and for actual practice.
Posted by: Anon | Mar 23, 2009 3:56:28 PM
I taught an undergrad law class (meant to give students a taste of what law school would be like) and refused to use multiple choice. In addition to being very hard to write so that there is an inarguably correct answer and three inarguably wrong ones, I felt it was too much of a bait and switch to spend the entire semester teaching them to question assumptions and make varying arguments, only to turn around and force them to pick out a single correct answer on the exam. Sure, they'll have to do it on the bar exam, but I don't like it and anyway, that's what BarBri is for.
Posted by: anon | Mar 23, 2009 2:37:29 PM
I've never, ever been asked to take a multiple choice exam in law school. The closest I've come is a very few exams that have asked me to to give yes/no answers followed by justifications of my answer in fewer than 50 words. If a professor gave a multiple choice exam, I'd immediately think less of him/her and less of the class. Isn't one of the main points of law school to teach students that very few legal issues are black and white?
Posted by: Anon | Mar 23, 2009 2:29:50 PM
I am in favor of only partial multiple choice exams (or no multiple choice). I prefer to write down the thought process of my answer and explain my points in writing. Essay writing also improves clarity and writing in a pressurized environment, more similar to real-life experience compared to multiple choice.
Posted by: Seton Hall Student | Mar 23, 2009 2:02:11 PM
The comments to this entry are closed.