« More on Oxbridge | Main | How Big Is Your Electronic Footprint? »

Sunday, May 13, 2007

In-Class vs Take-Home -- and Splitting the Difference

Like Dan Solove, I have grown increasingly dissatisfied with the in-class essay exam.  But I do think it has a virtue that Dan rejects: forming quick memoranda on legal problems is, in my limited experience, central to the practice of law.  Although lawyers do tend to get more than 3 hours to write up formal briefs, associates in law firms are routinely given only a few hours to research and write up a provisional solution to a legal problem.  In short, I think "issue-spotting" over the course of a very compressed time period is not just an exam skill, but one that lawyers actually use.  So I don't think wholly ditching the in-class essay exam is pedogogically sound.

On the other hand, it is far from pleasant for me to have to read 80 stream of consciousness papers and I recognize that the quick thinkers aren't the only sort that should be rewarded in the first year of law school.  Giving credit to students that do better with more time to think and organize seems appropriate too. 

So I'm trying an experiment this year.  One-third of my final is an in-class short answer exercise, one-third is an in-class issue-spotting essay, and one-third is a take-home essay with a 1500 word word-limit.  I feel this may help reward all sorts of students: those who are good at quick-fire spitback after careful studying; those who are good at the quick thinking issue-spotting exam; and those who can perform well if given time to develop ideas and execute an explanation with care and sustained attention.  Ideally, the A should go to the well-rounded student who can do all three reasonably well. 

We'll see how it goes.

Posted by Ethan Leib on May 13, 2007 at 03:37 PM in Life of Law Schools | Permalink

TrackBack

TrackBack URL for this entry:
https://www.typepad.com/services/trackback/6a00d8341c6a7953ef00d83514a35a53ef

Listed below are links to weblogs that reference In-Class vs Take-Home -- and Splitting the Difference:

» Some thoughts from the web about exams and more... from CALI's Pre-Law Blog
Some bloggers have recently posted some random thoughts about the whole one final exam tradition in law school (here and here). It's some pretty interesting stuff. You'll find this a lot with law school, though. People complain and complain about the ... [Read More]

Tracked on May 18, 2007 12:46:15 PM

Comments

I'd like to make just one quick point about incremental grading. In my experience as a first year law student, I haven't felt anything resembling comfort, much less mastery, of a subject until just prior to finals. Most of what we learn here is interstitial in nature and the material doesn't really hang together until all of it is on the table. Alternatively, reading an opinion the second time around (after the close of the semester) has far greater intellectual value for me than reading it the first time around, particularly if it was used as a case study in the beginning of the course. All of which is to say, in my humble opinion, that grading incrementally throughout the semester would be an absolute waste of time. I, for one, need time and exposure to a wide swath of materials in order to get a handle on this mess we call the law and make the kinds of connections that make it manageable. My measure of success in a course (though this may not translate into actual success) is the ability to speak confidently about something, to be able to converse about areas that weren't covered in class but that clearly relate and to be able to make those connections freely and with a sense of ease.

That said, why haven't law professors considered the prospects of an oral exam? Say, spend a half an hour or so (in a coffee shop or whatever) just talking and getting a feel for how well the student really knows their sh*t? I'm not making this suggestion seriously, but given the context, why not?

Regardless, I think the hybrid approach Professor Leib uses is fair and balanced (no pun intended). I know that people with family (or other) obligations have a harder time and are at a disadvantage relative to those without such obligations. As such, I think that testing this way levels the field a bit, and provides a means of evaluating performance at a number of levels. A professor of mine said that what exams really test is judgment (alongside the claim that judgment is really something that we can't learn, either we have it or we don't; this was a bit harder to swallow though I think it's probably true as well). The rules and everything else are important, but what law professors hope to cultivate is sound judgment and the ability to reason through discrete problems as advocates. My sense is that the hybrid-style might test this better than the alternatives, even if I would personally prefer a straightforward take-home.

Posted by: Frank Petrilli | May 14, 2007 11:31:15 PM

One quick comment only: I think one of the things law school evaluation does not do well at all is test students' abilities to integrate complex fact patterns into the legal rules they learn. Our exams -- even, I suspect, our take-home exams -- tend to be pretty sketchy on facts; surely they don't have page after page of deposition transcripts or product specifications or whatever. The result is we tend to focus on students' ability to spot issues and apply fairly basic rules to skeletal fact patterns. That's not the worst thing in the world, especially in a survey class, but it's not what I remember of practice, where briefing and memo writing (presumably the lawyering skills closest to those tested on exams) require an ability to spot relevant facts and integrate them into the given legal framework.

Posted by: Bill Araiza | May 14, 2007 10:05:17 PM

I don't think anything I said could be considered a complaint: I was suggesting that since I have to read 80 papers anyway, I'd prefer to read better organized ones rather than stream of consciousness ones. In any case, as you concede, the point of the post was not to complain but to think through the best methods of evaluation.

You do ask an important question: Why not have multiple evaluations throughout the term? As the interesting comments have revealed here, students have all sorts of reasons to like the one-shot deal: it is more disability-friendly; it creates only one period of stress rather than many; the school gives time off to study, which can't be done effectively in the middle of the semester; etc. Thus, your question doesn't produce an obvious answer in favor of multiple tests even from the student standpoint.

From the professor standpoint, there are a few pedagogical reasons that it doesn't necessarily make sense to evaluate students more than once. First, assessing whether students understand one piece of the course in isolation from the full set of issues in the course is not necessarily testing a very useful skill. Second, you basically have to waste two classes of the 28 in a given term to give a midterm; that affects coverage. Third, the stress levels get very high and your colleagues resent that students stop doing reading for their other classes. That has pedogogical implications for your course as well as other courses in which students are enrolled. When briefs are due in writing classes mid-term, we barely see our students. Adding to absenteeism is hardly a great solution from the perspective of pedagogy.

I do think, however, that the benefits of early feedback are very important. For that reason, I try to have reading responses or optional essays throughout the term. The students get feedback on their writing and my expectations without having to worry about the stress of a grade.

Finally, Jim, your attitude about the sweet life we professors have is a common one. And it would be absurd for us not to concede that we have a much better life than most. That said, adding to the grading responsibilities of professors IS a big deal. Teaching, when done well, is time-consuming. Preparing for class, when done well, is time-consuming. Advising students with their writing projects, when done well, is time-consuming. Answering non-stop student questions that come over e-mail is time-consuming. And of course we get paid, in part, to do these things. But part of the value we give students is contributing to the value of their JD by enhancing the academic reputation of the school. We do this through research and writing, activities that are always in competition with our more direct student-based responsibilities. But don't lose sight of the fact that when professors jealously guard their time to write, they are also helping students get the most out of their educations.

Posted by: Ethan Leib | May 14, 2007 2:00:12 PM

I did not mean to suggest that your post was merely a complaint about grading exams. Your questions about the best way to evaluate knowlege are valid, as is your point that grading is hard to do well. But you did give the deariness of grading papers as a reason to avoid giving them -- is it unfair to characterize that as a complaint?

When May rolls around and professors blog about law school exams, their posts always seem to mention that grading student exams is time-consuming and tedious. It's not just blog posts, either--all of my professors have complained to some extent about having to grade papers (even the professor who gave us a mostly multiple-choice exam that his assistant would grade). No doubt that's true, and perhaps we should call these "observations" rather than "complaints." But these observations seem to forget (from this side of the lectern anyway) that being a law professor is a pretty sweet gig overall. If spending long hours grading papers twice a year is the worst part of your job, thank the stars or your diety of choice you don't actually have to work for a living.

Back to the main point of your post, I wonder why incremental methods of evaluation are so disfavored over the all-at-once evaluation. Even your short answer/essay/take-home combo is a final exam. Why not spread the work and the grading out over the semester? Is there a pedagogical benefit to having an entire semester evaluated all at once when it's over, or is it just that it's too painful for a professor to grade papers or quizzes throughout the semester?


Posted by: Jim | May 14, 2007 12:49:43 PM

Just to offer another perspective, I've never offered a take-home exam. Many students also have family obligations at home that make it difficult for them to reserve a full 8 hour (or, god forbid, 24 hours) block of time to devote to taking an exam. Indeed, even if you have caregiving assistance, taking that kind of time away from family responsibilities is really impossible. Thus, in my view, the take-home exam advantages students who don't have these kinds of responsibilities. To make it fair for everyone, I give a four hour in-class exam.

Posted by: Anon | May 14, 2007 11:25:58 AM

A two-hour exam is kinder than a three-hour exam, but not 33% kinder. It's probably somewhere more like 5% or 10% kinder. There are certain fixed costs of stress, as it were, that apply to any timed in-class exam. Sadly, I'd imagine that your students resent you for giving them more overall things, even if each part is shorter. One way to take away some of that stress is to split up some of the topics between components. Carol Rose did this in her into-to-IP class the year I took it: we had a take-home essay on a copyright question, and a two-hour in-class exam on patent and trademark. The class consensus was that taking copyright off of the in-class exam was far far kinder than taking the third hour off.

Posted by: James Grimmelmann | May 14, 2007 8:58:08 AM

In my first-year civ pro class, I use a 24-hour take-home exam with strict page limits. I no longer use timed in-class essay exams in any of my classes. Part of this is personal preference--I liked and did better on take-home exams when I was in law school, so I carried that preference over when I began teaching.

My goal is to remove the immense, unusual time pressure and give students the chance to think, organize, and write clearly, while still requiring them to spot issues and analyze in fairly quick order. I generally am happy with the results. I think I get a better quality of paper overall, certainly better organized and written. It is easier for me to read these papers than the stream-of-consciousness papers that Ethan mentions at prawfs), which is nice. I think it is easier for me to evaluate such papers--I can better glean what students know from a less-rushed and (hopefully) better-organized answer because they have been able to spend putting forward a answer that best shows what they know.

I have encountered three problems, although I think many of these objections are a product of the relative uniqueness of the format in law school. First, ironically, students do not see this format as removing the time crunch--they see it as creating a new one in the need to work for 24 hours. And many say they would prefer the standard format, although I think that is a matter of familiarity rather than actually liking that format.

Second, the tendency among students is not to prepare as thoroughly before the exam, by outlining and reviewing their outlines and starting to understand how the pieces of the class fit together. The tendency among students is to think they can wing it as they go, using the full 24-hour period to outline, bring everything in the class together in their minds, AND write the exam. This becomes obvious in some of the weaker exams.

Third, I think not enough students do (or can) take advantage of what the format allows them to do: Spend a good deal of time thinking and organizing before they begin to write, then write to the page limit and go back and edit and revise and tinker. All while taking breaks to think and process . . . and relax or even sleep, which you can do in 24 hours. The result is more stream-of-consciousness papers than I would like or expect, as students follow their ordinary exam-writing process.

I have no plans to change my approach because I generally am happy with this. But Ethan's idea of including an in-class component, in addition to the take-home, is worth considering. The extra feature -- besides helping students with different skills -- also allows for more course coverage in the exam.

Posted by: Howard Wasserman | May 14, 2007 7:33:15 AM

Mr. Leib, Two points regarding disabilities.

At some schools, so many students are claiming disabilities, that it really creates two classes of students: those that had the foresight to go though the process, and those that didn’t.

Unfortunately, in the “real world” people get sent to jail and get the death penalty if their lawyer can’t concentrate. Or, at a minimum, clients lose money. The legal practice has never really reconciled the need to accommodate disabilities with the fact that opposing counsel will, can, and perhaps should take advantage of those disabilities. In firms, when word gets out that someone has a disability, perhaps with the exception of a noticeable one, rivals will figure out a way to take advantage of it. So, while it is bad manners to physically assault a cripple (and a crime), some consider it considered “smart” to make it difficult for someone with some attention-reducing disorder to produce good work product, unless it reflects badly on the saboteur.

Secondly, and I really don’t want to pick a fight with Mr. Frank, I think that there is some disconnect between the way law is practiced by experienced attorneys with a high level of specialization, and many exams. In my experience, most lawyers have a narrow area of expertise. But, in that area, they know every case and pretty much all authority. People like myself also keep up on analogous areas of the law and any sorts of “structural” issues (i.e. jurisdiction, “metaphysical” assumptions of courts, and the like). Therefore, when something needs to be done in a matter of hours, it isn’t the time for original research. Sure, an associate (or even an intern/summer associate) may be able to pull a case dealing with some analogous authority, but it might actually be more nuanced than he would understand, and he might not understand how it fits into an ongoing conversation. So, such “new” research is used at one’s own peril. So, what is going on during “tight” briefing schedules is more along the lines of articulating a well-established position, and explaining how it resolves the current question.

Posted by: S.cotus | May 14, 2007 7:31:19 AM


The implications for administering a non 3-hour in-class exam to disabled students may be real -- and even rather substantial. But then it is the school's responsibility to design a workable solution for those students. It hardly follows that professors should retain sub-standard evaluation techniques for everyone else in the class merely because they are somehow moderately less cumbersome for adminstration to the disabled.

Posted by: Ethan Leib | May 14, 2007 1:55:19 AM

Professors who schedule alternative format exams often fail to take into account one reality of modern student bodies - increasing numbers of students have been diagnosed with cognitive processing disorders and routinely receive double time or time-and-a-half for exam taking. Thus, if you give an 8 hour take-home, these students get 16 hours, which is actually kind of cruel if they really need that much time. If you give both in-class and take home, students get double time in-class and then a separate double-time take home, which is a nightmare for scheduling. Moreover, a not insignificant number of students use scribes (people who write/type for them) or readers (people who read exams to them). Since these people are hired by the university's office of disabilities and aren't necessarily people you would take home with you, those students don't actually get to take their "take-home" home at all and just have to sit in a room in the law school all day doing the exam. Most professors who choose Ethan's style of exam because they think they are helping students and producing a more accurate sorting effect are really just throwing havoc into the administration and taking of the exam, while skewing the sorting effect so as to harm disabled students.

Posted by: anon | May 14, 2007 1:06:41 AM

Is that how students should be trained? To model their work product based upon that created by ID firms? Should we tell students to ignore statutes and cases as well, since, in real life, many attorneys ignore those things?

I would hope that a nobler goal is achieved by law school classes than preparing students to "think quick." I thus think your approach is a step in the right direction, but I would guess that the vast majority of exams today are given under severe time constraints, or, even more disturbingly, in "closed book fashion." It is bizarre and depressing that some professors do not allow students to take a copy of the Administrative Procedure Act with them to an administrative law exam.

Posted by: andy | May 13, 2007 11:06:06 PM

I know it is hard to believe. But such "malpractice" is not uncommon. You sometimes get a few hours to figure out a legal problem -- without the benefit of an entire 5 months devoted to studying the subject.

See this comment at the Co-Op:


"Anyone who thinks that briefs are never drafted (or rewritten by a senior associate or partner) in three hours or less has never practiced litigation in a large law firm. I was frequently responsible for turning around local counsel's work in three hours or less because of the time constraints of filing and the frictions of coordination among the client, national counsel, and local counsel, and that did take some real high-pressure rewriting from time to time that made my in-class law-school exams look like a piece of cake. I've been in rocket dockets where reply briefs need to be filed within 48 hours of the 5:00 pm fax of the opposition brief, and the logistics of getting approval from the senior partner, co-counsel, local counsel, and client (and the logistics of putting together exhibits for filing) mean that the first draft of the brief needs to be out the door that first night. I've been in trials where the phone call came at 10:00 AM that an issue had come up and an evidentiary brief was needed to give to the judge by the 12:30 lunch break. And the attorneys who worked more frequently with temporary restraining orders had it far worse than I did.

And I had friends doing insurance defense law firm work where they had to write briefs in three hours because the insurance company wouldn't pay for any more time than that. And I have to imagine public defenders write their briefs under heavy time constraints, too, simply because their caseloads are so heavy that the choice is a three-hour brief or no brief at all.

Posted by: Ted F. at May 13, 2007 08:37 PM"

Posted by: Ethan Leib | May 13, 2007 9:34:41 PM

"forming quick memoranda on legal problems is, in my limited experience, central to the practice of law. Although lawyers do tend to get more than 3 hours to write up formal briefs, associates in law firms are routinely given only a few hours to research and write up a provisional solution to a legal problem."

If an associate (or any lawyer) solved a client's problem in the way that he prepared an exam answer, he would be guilty of malpractice.

Timed exams may have several virtues, but simulating the real-world is not one of them.

Posted by: andy | May 13, 2007 9:13:41 PM

The two comments above should helpfully clarify part of the problem: for all the extra work professors want to create to help improve the evaluation process, students rebel because it often means more work for them too.

My current plan does not, I think, brutalize students. They have time to write 1500 words (1500 words is not especially long) -- and their in-class exam is a merciful two-hour test.

Developing useful evaluation tools is very hard work. Grading is not only boring -- but it is also hard to do well and to do fairly. These are not mere "complaints," Jim. These are realities that are underappreciated. If professors were really committed to doing as little grading as possible, there are easier systems than the ones we have. The ones we have cannot be explained by the principle you imagine. Law professors are some of the few professors in the academy who actually do their own grading.

Posted by: Ethan Leib | May 13, 2007 6:54:14 PM

Might be better for evaluating their skills, but it's pretty brutal to the students who now have to do all the studying for an in-class plus all the writing and careful editing of a take-home (as well as fit their other classes in somehow).

Posted by: anon | May 13, 2007 6:10:59 PM

Ah, May -- the month when students complain about having to take exams, and professors complain about having to grade them.

Why try to cram all three methods of evaluation into the single mechanism of the final exam? Surely graded homework assignments, semester-long writing assignments, or in-class quizzes would also reward some of those other than the quick issue-spotters. These would also have the side benefit of letting students know how they're doing before it's too late to do anything about it. But that would conflict with the real (if unstated) primary goal: don't make the professor grade more than he or she has to.

Posted by: Jim | May 13, 2007 6:07:11 PM

Post a comment