Sunday, November 01, 2009
Exams, Good and Bad
I'm delighted to begin another month of blogging here at Prawfs; as always, I'm very grateful to Dan for providing a soapbox.
I want to begin my stint with a reprise of a topic I blogged on some time ago: final exams. It's about that time of semester for profs and students both to start thinking about them; thus, it seems a good time to ask again for thoughts about good and bad exam drafting. I'm interested in perspectives from both sides of the podium. What have you seen that works, and that doesn't? Was there a particularly good or bad/ fair or unfair exam that you took or drafted? What made it so? For students, if you could tell your teacher one thing about drafting the exam, what would it be? For profs, if you could change one thing about how students write their exams, what would it be?
This is a "fun" topic, but it's also of course quite serious. We all know exams are make-or-break events for law students, as much as we might lament that fact. If we're going to do high-stakes testing at the very least we should engage in some dialogue about what works best.
Posted by Bill Araiza on November 1, 2009 at 11:02 AM | Permalink
TrackBack URL for this entry:
Listed below are links to weblogs that reference Exams, Good and Bad:
I appreciate all these comments, and I hope they're useful for other profs as well. I have to say I'm always surprised to hear about questions that ask a student to answer a question as if it were some point in the past; surely there are better ways to test whether a student has understood the historical development of a rule. I also appreciate Orin's comment about using obscure words or terminology, and "wrote some's" complaint about absurd (and thus maybe distracting)fact patterns. In some basic sense both comments speak to the reliability of the exams we administer. I do wish we would pay more attention to these issues, rather than treating exams as something to be dashed off a few days before they're due because we've put off drafting them until the last minute.
Posted by: Bill Araiza | Nov 4, 2009 9:44:19 PM
I always really appreciated exams that clearly allocated points among questions and gave suggested allocations of time/words. Especially for tightly-timed exams, it was immensely helpful to have some guide to help you pace yourself that reflected the weight to give each question rather than simply the length of the hypo being your guide. Though, at the same time, very tight word limits for exams that are given less than eight hours for completion are frustrating because some students end up spending more time cutting down words than making sure answers are complete. For example, I had a six hour "take home" exam (who can go home and get back and do a good job in six hours?) where 5 of 6 questions were limited to 500 words and the 6th was limited to 200 words. Most of the fact patterns were significantly longer than the word limits for answers (2-3 times as long). It was otherwise a reasonable exam, but allowing something like 1.5 times as many words would have meant we all could have spent more energy on complete and correct answers.
Finally, I think for upper-level classes it is frustrating to have exams in doctrinal-type classes that have questions asking you to apply old, out-of-date doctrines simply because your class had to spend time on the historical development of the field. Something like that might be appropriate for a 1L class (though maybe still questionable there), but it's kind of ridiculous to have an exam question in say Education Law where it begins "It's 1965..." and you are expected to apply the law that was applicable then for a school desegregation case.
Posted by: No longer a student | Nov 2, 2009 9:01:29 AM
my pet peeve: absurd fact patterns with wacky aberrations, things that don't happen in real life, etc. an expert in a legal field ought to be able to construct an appropriately difficult fact pattern that persons versed in that art would recognize as being relevant and relatively tough to analyze.
Posted by: wrote some tests; took some | Nov 1, 2009 6:33:37 PM
Orin - I encountered a similar situation on a law school test. In a Land Use class the fact pattern was based in Vail, Colorado. I don't remember the entire context, but the gist of the location relevancy was that there were resort and snow skiing considerations at stake (not outlined in the test, you were to infer this). While I would like to not come off as a complete rube, at the age of 24 this may not have been an unfair characterization. In fact, i t was only in that same year that I had ever traveled outside of the south or been on an airplane (both for moot court). I suppose I should have asked about it, but I did not, and my answer missed the mark - oh well. :-)
Posted by: Jeff Yates | Nov 1, 2009 12:57:23 PM
In the opinion of this mere law student, a good exam:
1) Tests the students' knowledge of the material covered in class,
2) In a direct way (this requires professors to distinguish hypos that actually get at critical issues from hypos which are largely flights of abortive literary ambition),
3) Strongly weights the assessment of mastery of substance or large points of law over the memorization or mastery of trivia or minutia (my favorite example of this is a professor who gave credit for knowing the name of the particular judge who decided one of the 50 cases read in the class--and nobody could recall any discussion of that judge as such in class).
4) Does not require students to assume any relevant hypothetical facts not on the page in order to answer the question. One way in which professors violate this rule is by giving a multiple choice question whose answer will be different under different conditions--but the question itself doesn't specify which conditions obtain.
5) Is written by a professor who remembers that the tests are timed (and that it takes a long time to carefully read a 4 page hypo). The professor should decide whether the curve is more appropriately based on a) ability to do a serious rush job, or b) knowledge of the material. It might be that a and b converge, but it might not, and so the question is worth keeping in mind.
Posted by: Michael Young | Nov 1, 2009 12:26:03 PM
Here's an obvious but important point about exam drafting: Don't use words that some students might now know. In the first exam I took as a student, Criminal Law, the first question was an issue spotter in which a defendant entered into a "vestibule" to commit a crime. The issue was whether the "vestibule" was a building or occupied structure for the purpose of the burglary statute. I had a vague sense of what a "vestibule" was, but I wasn't sure if there was some subtlety in the meaning that I wouldn't pick up to answer the question correctly. And I remember thinking it was pretty unfair that how well you answered the question was based on your knowledge of a somewhat uncommon word rather than your ability to apply the law.
Posted by: Orin Kerr | Nov 1, 2009 11:43:27 AM