Monday, May 14, 2012
On Timed Exams
Every year I teach Disability Law, I spend at least one class on exam accommodations. We talk about accommodations for people with physical disabilities, vision disabilities, and, of course, learning disabilities and ADHD. It won't surprise anyone that the accommodation that generates the most discussion is the provision of extra time to take an exam. In fact, my casual empirical conclusion is that testing accommodation is the area of disability law that draws the most interest from both students and faculty at law schools across the country. Basically whenever I give a talk at a law school, some faculty member engages me in a discussion of whether it's appropriate that some students with some disabilities get extra time on their exams.
Still, I must say that I was a bit unprepared for just how much discussion the topic provoked in my Disability Law seminar this past semester -- and how upset so many of the students in my class seemed to be about the number of students (whom they believed to be) getting extra time accommodations. It wouldn't surprise me that the current economy and job market anxieties are making students more concerned about anything that might give their colleagues an unfair advantage (and at the same time making students more inclined to seek an advantage by obtaining extra time themselves). Also, the passage of the ADA Amendments Act of 2008, which overturned a series of Supreme Court decisions that narrowed the definition of disability, means that more ADA challenges to the failure to provide testing accommodations are likely to be successful than in the past. But, in any event, the discussion of the issue, and the level of skepticism of testing accommodations, was more fierce this past semester than at any time I can remember in the dozen or so years I've been teaching Disability Law.
Many of the people I talk to about this issue (including a lot of faculty and a number of my students this past semester), see two basic, mutually reinforcing, problems with extra-time accommodations on tests. First, many students and faculty believe that everyone would benefit from extra time on their exams, so that even for people who legitimately are diagnosed with learning disabilities the accommodation gives them an unfair advantage. Second, many students and faculty believe that the criteria for a learning disability or ADHD diagnosis are fuzzy and that students of greater financial means are more likely to be able to find a clinician who will make such a diagnosis -- so that people who don't really "deserve" extra time will get it, and that the extra time accommodations will, if anything, reinforce existing inequities. This leads lots of folks to conclude that we shouldn't give extra time on law school tests, at least for people with learning disabilities and/or ADHD.
I sympathize with some of these critiques. But I think the problem isn't with the testing accommodations so much as it is with the tests themselves. More below.
First of all, I think it's clear that on most law school exams, everyone would benefit from extra time. There is a dispute in the testing literature regarding whether people with learning disabilities will benefit more than others from extra time. But my sense from talking to many students and faculty members in a number of law schools is that basically everyone feels time pressure on most law school tests. What that means is that our exams don't just test for understanding or mastery of the substantive material; they also test for speedy processing -- the ability to formulate and provide a response quickly. So, as Mark Kelman and Gillian Lester argue, on most law school exams giving a student with a learning disability extra time isn't like giving a blind student the exam in Braille (nor, to adapt the facts of a recent case, with accessibity software like JAWS and ZoomText). There is a real sense in which extra time makes the exam "easier," not just "different." Especially if learning disability or ADHD diagnosis is fuzzy, and/or easier for wealthier students to obtain, this means we really should be concerned about extra-time accommodations on law school exams.
Now, a lot of people stop there and say that the solution to this problem is to stop giving extra-time accommodations, or to give them a lot less frequently. But the key question, it seems to me, is why we give such time-pressured exams in law school to begin with. There is no doubt that speedy processing is one important skill for a lawyer. Most lawyers will have to think quickly "on their feet" in a variety of settings: court proceedings, negotiations, client presentations, etc. Sometimes litigators have to write emergency motions. And lots of lawyers have to bill a large number of hours and be sufficiently productive during that time to justify a high hourly rate.
But my sense, from having practiced law and seen a lot of law school exams, is that our exams vastly overvalue the skill of thinking quickly on your feet relative to the importance of that skill in practice. For many issues in most practice settings, time is, within reason, on your side. Most briefs don't have to be written in a couple of hours, most research and discovery projects don't require the sort of speedy processing that our exams tend to test for, and virtually nothing in legal practice much resembles the act of writing a memo in an hour and a half identifying issues in a complex fact pattern. The skills of thinking through different and conflicting lines of precedent and developing legal theories and litigation strategies are often quite important in legal practice, but the ability to do those things in a three-hour period is far less so. Yet on our exams we highly value the ability to think through these issues quickly -- even though in most practice settings you would never rely on a synthesis of precedents or legal strategy developed in only three hours, and the people who can come up with the best synthesis of precedents or legal strategy in a three-hour period are not necessarily the same people as those who would come up with the best answers over a few days of thinking about the problem. (Obviously there are differences across subjects here. I would think that an evidence class should value speed more than a jurisprudence class, to take an extreme comparison.)
If, as I think is often true, our examinations overvalue processing speed in relation to the importance of that skill in legal practice, I believe that the solution is to give all students more time. For this reason, I give take-home exams wherever possible. Particularly with 48-hour take-home exams, I believe that every student will have an ample amount of time to demonstrate his or her skills, and no student will need extra time. This avoids overvaluing speedy processing and avoids any potential unfairness of giving some students, but not others, extra time.
Yet I often find that colleagues are resistant to giving exams of this length. In part, this reaction reflects a legitimate concern with the increased potential for cheating. In part, it reflects a compassionate if paternalistic concern for the students who will work constantly on the exam for 48 straight hours. But I fear it also reflects at least in part an unexamined assumption that the way we have always done things must necessarily be the best way. I wonder what folks who read this blog think, and whether they have any better ideas than I have come up with.
I've blogged a fair amount on these issues over at my Disability Law Blog, if anyone's interested.
Posted by Sam Bagenstos on May 14, 2012 at 02:55 PM | Permalink
TrackBack URL for this entry:
Listed below are links to weblogs that reference On Timed Exams:
The 48-straight-hours thing seems like a very important and not paternalistic at all concern. Not paternalistic because the problem isn't students who would voluntarily work on the exam for 48 straight hours, but the competitive pressure forcing all students to do so.
Posted by: Paul Gowder | May 14, 2012 3:11:01 PM
Fair enough. It's certainly an empirical question, but I don't think that anyone will be helped by working 48 straight hours on one of my exams. I try to write an exam that could be answered in 4 to 8 hours and give the students 48. The point of zero marginal returns will kick in well before 48 hours of work for basically all students, I think. In which case, the concern is paternalistic, because some students may think they will be helped by working 48 straight hours when in fact they won't. But I could be wrong about how much work my exams require, to be sure.
Posted by: Sam Bagenstos | May 14, 2012 3:16:50 PM
A contrary concern raised by some is that for students who have children at home, a 48 hour exam actually puts them at a disadvantage versus those without children. My preference is actually the model from when I took them at HLS - 8 hour exams. It's more difficult administratively, however, to manage.
Posted by: dave hoffman | May 14, 2012 3:22:52 PM
I tried 24 hour exams, but the students hated them so much that I stopped (perceived competitive pressure to stay up for 24 hours). Maybe it would be better at 48, because you'd have to sleep, but I still think students are likely to feel required to give up a lot of sleep for that. Now that computer downloads are possible at Georgetown, 8 hours is much easier to administer, but I am still tempted by the thought of longer ones. What about the entire exam period, which I know some people use?
The only problems are (1) conflicts with other courses, and (2) for first years, I've found our registrar quite resistant to anything other than everyone taking the exam at once.
Posted by: Rebecca Tushnet | May 14, 2012 3:41:06 PM
Different style exams test different things. A three-hour exam tests students' knowledge of the material in a way that a longer examination cannot do, and should benefit students who have done the work throughout the semester. But if you want to test skills that are likely to be most useful in practice, then a longer exam will surely be better at that, though I would think a week-long take-home would be a better approximation than 48 hours.
Posted by: MS | May 14, 2012 3:48:29 PM
Attorney: Objection, Your Honor.
Judge: Huh? What are you objecting to?
Attorney: The question my learned adversary asked yesterday, just after lunch.
Judge: I think you're a bit late on your objection, counselor.
Attorney: Well, Your Honor, perhaps you aren't aware of this, but even though I was first in my law school class, I'm a slow processor.
Judge: And I care about this why?
Attorney: I am allowed extra time, Your Honor. I was at Harvard, and certainly this Court doesn't consider itself more worthy of my processing than Harvard.
As long as you can provide an absolute assurance that no student who receives extra time because of slow processing ever steps foot in a courtroom or other place where he could compromise a client's interest, since all lawyers are admitted as generalists and otherwise have the license to do so, it's good with me.
Otherwise, neither your concern nor that of your slow processing student trumps the interest of the client whose life, fortune or rights are lost in the name of accommodation. We don't practice law for our own benefit, but for the benefit of our clients.
Posted by: shg | May 14, 2012 3:49:15 PM
I basically agree with the main point and have shifted to longer take-home exams to remove the advantage of speedy processing. However, there is one big issue you've omitted: the bar exam. Students taking the bar exam in more challenging states and students whose statistics put them at risk for bar passage (and some law schools contain many such students) need practice taking timed exams. Beyond that, the ability to identify those students who lack that particular skill is *crucial.* It is unethical to allow law students to pay tuition and spend 3 years only to discover that they are not able to perform the exam-related skills on the bar. So any movement to change the fundamental nature of exams in law school does need a method for simulating and practicing bar testing, until the bar exam itself catches up to the skills necessary for 21st century law practice.
As for how long to make the exam if you do use the longer take-home method, that's something I always struggle with. 48 straight hours as a default wouldn't tend to work well, because:
(1) It penalizes students who work and/or have family responsibilities that do not allow them to devote as much time;
(2) Most law schools operate on a compressed time period - all exams occur in a short amount of time & short study period before, so students carrying a full load would be in "exam time" for the equivalent of 8-10 days, with limited time to study or decompress; and
(3) It encourages "exam creep" where more professors are giving take-home exams while other classes are still meeting, which can be very frustrating for the professors who are teaching (this already happens and would only get worse).
I have used time frames from 4 to 24 hours and find that there is a point somewhere in the middle of that where the diminishing marginal returns are clear, if you write an exam that could be answered in 3 hours by the "hypothetical reasonable speedy test taker."
Posted by: Anon Prof | May 14, 2012 3:56:58 PM
When I was a law student at Harvard, I heard rumors of cheating on 1L exams that used the 24 hour or 48 hour take-home format. I have no idea if the rumors were true, but they were enough to discourage me from using that format as a professor.
Posted by: Orin Kerr | May 14, 2012 3:57:30 PM
Perhaps. I think that's really dependent on the same premise as Paul's point, though, no? That is, if the students get 48 hours to take an exam for which they really need to spend only 8 hours, the student with kids at home shouldn't be disadvantaged. And, in fact, the student is plausibly better off as compared with the Harvard 8-hour take-home. What if your kid gets sick right at the beginning of the 8-hour period? You'd be a lot happier with the flexibility the 48-hour exam gives you. But, as with my response to Paul, I admit I might be wrong empirically. This is just my sense.
The work-all-night problem is precisely why I wouldn't give a 24-hour exam. But I do think 48 hours should solve the problem, as it's not reasonable to think that you should work for two days and nights straight, and in any event most people will just be unable to push themselves that way so will figure out a time to sleep. I like the whole-period exam, too, but have found the same administrative obstacles you have.
I agree with your point entirely. I don't think there's any one way to test that perfectly captures everything that it is valuable to test. But I think that the predominance of 3-, 4-, and 8-hour exams means that we are systematically overvaluing speedy processing. Of course, a big part of the problem here is the one-exam-at-the-end method of evaluation that we use in big law school classes. We could test in different ways throughout the semester.
I am tempted to say that you didn't read my post closely enough, but I have a lot of respect for your own work so I am sure that I just didn't make myself clear enough. As I said, speed certainly has value for lawyers. The question is whether our testing practices *over*value it. And, as I also said, speed is more or less important depending on the subject. I specifically referred to evidence as a subject for which speed might legitimately be thought especially important (and thus given especial weight in testing).
But: First, it seems to me silly to say that because lawyers in court will be forced to make evidentiary objections quickly we should give great weight in *all* of our testing on *all* of the major subjects in law school on the skills that correlate with the ability to make evidentiary objections quickly. (The point holds true even if we say *most* of our testing on *most* of the major subjects, I think.) Someone who is really quick on her feet in making evidentiary objections may nonetheless quite poorly serve her clients if she cannot engage in the more deliberative process of developing a theory of the case and an investigation, discovery, and litigation strategy. The attorney who does well at those more deliberative tasks may be more valuable to clients than the attorney who does well at the more on-the-fly tasks. This is especially true given the very high proportion of litigation that does not culminate in a trial. (And that's just in litigation. Lots of lawyers aren't litigators.)
Second, even in the field of evidence, lots of the law of evidence is applied not in objections made on the fly, but in motions in limine, which often (but not always) involve a reasonably long gestation period. Indeed, as you know, the law of evidence is very important outside of court, when a lawyer is figuring out what evidence to try to obtain in investigation and discovery -- decisions that again are more deliberative.
Posted by: Sam Bagenstos | May 14, 2012 4:17:09 PM
I probably cannot convince the professors who discount/dislike the traditional exam format, but I do have a consideration I believe merits discussion. That is one of the "dump truck" approach to questions.
If I provide a limited time period to answer a question, I know everyone will give me their best analysis right away. If I give them 48 hours, I suspect that many will provide their best analysis right away, but others will apply a "scattershot" approach. (to those who havent graded, this si common on a 3-hour exam as well, but necessarily limited).
Since all students will feel pressure to write as much as possible (to "keep up with the joneses"), I see it becoming extremely difficult to grade and regrade essays that offer five alternative explanations for each subsection of analysis.
To the extent the answer is word limits, then I ask: why not have a defacto word limit based on time? A three-hour exam necessarily involves some limitations on space/length, and so why am I wrong to use that limit?
To me, this seems much ado about nothing. Maybe 48 hour exams are OK for you, and if done right, a good alternative. But they have down sides too. And if I can write a 3-hour exam correctly, why is that bad again?
Posted by: Anon | May 14, 2012 4:29:01 PM
Sometimes I try to make a point via humor (and used evidence as an example this time. Next time, I'll use torts). I addressed a slice of your post that you cruised past rather quickly. There are no doubt areas where leisurely testing is appropriate, but my concern is about the student who can't process quickly enough to take a normal test being allowed to proceed in law school. I am similarly concerned about students with other deficits. They may be lovely people, but the only ones who should be coming out the other end of law school are the ones who should be lawyers. If you figure out that one lacks the abilities early, do them (and clients) a favor and hand them a dime.
Posted by: shg | May 14, 2012 4:46:34 PM
I find this discussion about exam format interesting. On the original question, however -- whether giving extra time on exams is fair -- the consensus seems to be "no." The conclusion to draw is that while eliminating time pressure for everyone is the optimal solution, providing extra time as an accommodation to the disabled is never appropriate. It is equivalent to giving them easier tests.
Posted by: AF | May 14, 2012 5:01:25 PM
"Particularly with 48-hour take-home exams, I believe that every student will have an ample amount of time to demonstrate his or her skills, and no student will need extra time."
At every law school I have taught at, the administrators have permitted a student who needs twice the time as part of their exam accommodations to get twice the time even in an 8 or 48 hour exam. So, the student gets 4 days rather than 2 days in your example. Do you somehow override disability accommodations for such students when you give a 48 hour exam in your law school or are you simply saying that the non-accommodated students should feel less aggrieved. If the latter, that presents the line-drawing problem, since surely there will be some students who spend 48 hours on the exam and feel that they could have used an extra day or two, just like there are students on what the professor feels like is a 2.5 hour exam, who need even more than the 3 hours provided. At probably every length of exam imaginable, professors give exams where they feel time pressure isn't an issue for students who have studied the materials and yet that have students who disagree.
Posted by: Anon | May 14, 2012 5:54:49 PM
I gave a 24 hr. take home exam for my Jurisprudence class this semester and it seemed to work fairly well. I stressed, several times, to my students that there was meant to be about 3 hours worth of material in the exam, and that it really wouldn't do them much good to spend much more than that on it. What they could do is have more convenience (they could pick up the exam at their convenience during the exam period) and to polish their writing. I also stressed that the "dump truck" method was going to cost them points. I had pretty tight word limits for questions, and told them that if they put down lots of stuff, I'd assume they thought it was all equally important, but as I knew it wasn't all equally important, they'd do worse by doing so. Thankfully, few people tried that method. (While clerking I saw many lawyers try that method in briefs, too, I should say. It didn't work for them, either.) It's possible that this wouldn't work as well for some classes as it did for jurisprudence, but it seemed to work pretty well for this class. Anyone who tried to work the whole time would have just shown themselves to be dumb, as you couldn't really do better for try that.
Orin- what sort of cheating did you hear of? I assume that a take-home is open book. If not, it seems irresponsible. If there's a curve, there's a built-in incentive not to help others.
Posted by: Matt | May 14, 2012 7:22:56 PM
shg, there is lots of ways for lawyers to make a living outside the courtroom. I reckon a blind lawyer might be a problem there too, that is, if viewing witnesses is an important to the situation. Should blind people promise not to do that sort of thing before being allowed to get an accommodation during testing?
Posted by: Joe | May 14, 2012 7:39:31 PM
I used to regularly give a 24-hour take-home exam--and the university disabilities office insisted that students who receive double time on their in-class exams also were entitled to double time on this exam.
In several classes, I have moved to a combination exam--a timed in-class portion and an untimed take-home portion, with the latter distributed on the last day of class and due when they show up at the in-class (which often functionally means, as Rebecca suggested, having all or almost all of the exam period).
Word counts are better than time at limiting the amount of time students put into it. If I give a 24-hour exam with an 10-page limit, a good student should be able to write to the limit in about 10 hours, then spend some or all of the remaining time editing and, hopefully, resting.
Posted by: Howard Wasserman | May 14, 2012 8:01:02 PM
I would like to underscore the importance of word limits.
If you have a tight word limit, and give students a generous amount of time in comparison to the number of words, you shift the main constraint under which students are operating. Instead of just vomiting up as many correct points as possible onto the page, students facing a word limit that is stricter than the time limit will have to think more about priorities. They will have to decide which points deserve just a quick half-sentence (even though they could say more) and which deserve their own paragraph. To my mind, this has several important advantages:
1) Pertinent to our discussion here, it lowers the stakes in fights about "extra" time. I am unaware of any disability where the accommodation would involve extending a *word* limit. (Logorrhea?) So if the word limit is the real constraint, disability accommodations will not raise the problems they raise under the more common, time-constrained exam format.
2) At the same time, making the word constraint tighter than the time constraint helps eliminate a clear advantage that flows in the traditional 3-4 hour no-word-limit exam to those students who are "uninhibited" writers and fast typists -- those who can literally put more points on the page per minute because they do not feel the need to craft or edit their sentences. I doubt that this skill/approach is highly correlated with successful lawyering but absent a tight word limit, it confers a huge exam advantage. I also believe that this "uninhibited" approach (in which you do not even try to craft or edit your sentences) is likely to be more prevalent among male students than among female students, so that the traditional 3-4 hour no-limit exam format may have a disparate impact on the basis of gender to the detriment of female students. But my evidence for this is anecdotal and spotty.
3) Oh, and by the way, the skill of prioritizing one's points and fitting them into a limited amount of space is an important skill for lawyers, especially any lawyer who is ever going to write a brief.
In conclusion: set a tight word limit. (As a free bonus, it will also make the exams more pleasant to read.)
Posted by: Joey Fishkin | May 14, 2012 9:44:45 PM
"If there's a curve, there's a built-in incentive not to help others."
Collaborating with one or two other people (e.g. your girlfriend/boyfriend, or your tight-knit study group that was formed for the purposing of maximizing your grades) won't screw up the curve. I always assumed (uncharitably, i admit) that couples who took nearly every course together cheated on take-home exams.
Posted by: Doug | May 14, 2012 9:54:59 PM
I'm thinking about trying an approach next year that I think might be really interesting or really stupid (or both!). I would distribute the exam at the start of the school's exam period and then have various submission deadlines, perhaps at 6 hours, 24 hours, and 72 hours. Obviously, the word limits and my expectations would change depending on how much time a student took. All of the 6-hour exams would be graded together and only compared against each other; the same for the 24-hour and 72-hour ones.
The benefit is that it lets the student determine what her schedule will permit without worrying that "even though the professor says it's only a 6 hour exam, other students can and will use the full 24 hours."
The most obvious downside is that the registrar is going to kill me when I propose this idea.
Posted by: Michael Teter | May 15, 2012 12:05:22 AM
Joe - Physical disabilities are flagrant; if a client retains a blind lawyer, he knows his lawyer is blind. That said, there have been some great lawyers, from Judge Casey in SDNY to one of my best pals who has tried more than 200 felonies to jury verdict. They learn how to use their disability to their advantage.
That a lawyer can't process quickly enough to fulfill an entire swathe of the functions he's licensed to perform, on the other hand, isn't obvious to the client, and there's no requirement that he either limit his practice to things he can adequately do or inform his clients that he's less competent than others.
If every academic discussion included serious consideration of whether a choice enhances a student's ability to become a lawyer and serve clients, we would all be much better off.
Joey Fishkin - Excellent point about word limits, and using examinations to both test and reinforce a skill that's demanded of lawyers. That's exactly the sort of approach that moves a law student toward the skills he needs as a lawyer, and costs nothing in the process.
Posted by: shg | May 15, 2012 7:42:20 AM
Perhaps the students who are upset about these policies aren't convinced that there is much of a difference between "ADHD" and "not smart enough to score higher on the LSAT." Perhaps students believe that the only difference between those two "disabilities" is that one has a catchy name and is more likely to be "used" by pushy, entitled students who likely come from privileged families. In other words, the students believe that the ability to focus, concentrate, and reason quickly is just as much of an attribute to being a lawyer as the ability to reason through who is wearing the red sweater if the man to the left of the woman wearing the blue cap is etc etc etc.
Just a thought.
Posted by: Skeptical | May 15, 2012 8:01:46 AM
shg, the last point is fine as far as it goes, and my question seems to be useful since this "hidden disability" qualifier does not seem to have been clearly shown.
Two things. Clients don't know who exactly is serving their needs each time. If a business hires a law firm, the business probably does not know each and every time who is sent to court. Second, there are lots of disabilities that are not readily apparent. In fact, a person can have vision problems that severely hinder in a court setting but in an office setting, it might not be readily apparent.
The point is that there are plenty of disabilities out there and the disabilities can very well inhibit some types of law practice. It would be appropriate in various cases to disclose this fact. To the degree your example fits the bill, it is basically but one possible example of a lot of things.
Posted by: Joe | May 15, 2012 9:18:14 AM
Are employers allowed to ask applicants about disabilities? Certainly I never thought of asking while at Justice, but then Justice is large and wealthy enough to accomodate virtually any lawyer's disability in an important and fulfilling attorney position.
Posted by: jt | May 15, 2012 10:25:21 AM
Joe - That's why law schools need to vet their student populations for those, whether disabled or just not intellectually suited, who should not be lawyers. It's not an entitlement. It's not a pay-to-play. I can't, in a blog comment, expound on every potential disabilith that might impair someone's ability to practice law.
I would have thought this clear already, but I will spell it out for you: If a law students lacks the ability to serve clients as a lawyer, then he/she has no business continuing in law school. Every decision must pass muster under one overarching concern: How does this serve clients? It appears that some have forgotten that this isn't about law students, lawyers, lawprofs or judges. It's about serving clients. If we don't/can't serve clients, we have no reason to exist.
The public (including businesses) expects that law students entering the profession are competent, both intellectually and ethically, to hold their trust. I know, it's foolish and naive, but the public thinks that anyone admitted to practice as a lawyer is capable of doing so. Go figure.
If we had separate licenses that limited our authority to practice to those aspects where we are competent, but precluded practice in areas where we are not, then we could accommodate disabilities differently. But unless and until law licenses are limited or specialized, and as every lawyer is authorized to perform any legal function (as well as a bunch of others tangentially related to lawyering) possible, the responsibility of safeguarding the client must be paramount.
Posted by: shg | May 15, 2012 11:20:54 AM
Coming from a non-teaching career into the Academy this fall. I tend to agree with Sam's general points about timed exams not really reflecting real world situations, and thus not really testing well for real world skills.
Quick question: is there any reason (other than the added time it would take to grade) that one can't simply assign a research assignment, testing for various concepts covered in class, the way one would get asked to do this at a firm? What would be the drawbacks of this approach, as opposed to the more traditional issue-spotting exam?
Posted by: DKM | May 15, 2012 11:21:49 AM
Great post and great discussion. I have always given take-homes in Criminal Law and Criminal Procedure. At the beginning of my career, I caught some flak from more senior colleagues who were very traditionalist and set in their ways, but it did not stop me from getting tenure. I used to give the students 48 hours, then I switched to giving them the entire exam period, and I recently switched to 24 hours, but they can choose any 24-hour period during the exam period. Given Sam's comments above, I may go back to 48 hours. I give them a 12-15 page limit, depending on the exam.
I won't reiterate the points made above, but I would add that I have observed that take-home exams ameliorate what I call the "funnel effect" of in-class exams: students use the dump truck approach early on in the exam, then frantically try to sketch out their answers in outline form as the end of the exam draws near. The result is an over-analysis of easy issues at the beginning of the exam and under-analysis of the difficult issues at the end of the exam. Of course, the same can happen where there is a page limit rather than a time limit, but the critical difference is that if the student gets to the last page and realizes she has not addressed half the issues, she can go back and rethink things.
That is why I recently switched even my Evidence exam to a take-home. True, evidentiary arguments are sometimes made in the moment, but typically they are anticipated. In addition, a good many "evidentiary" issues are more fundamentally about the underlying substantive law anyway. And I was getting tired of students writing a page and a half on why a now-deceased victim's statement as to who killed him is relevant in a trial for murder.
Posted by: Michael J.Z. Mannheimer | May 15, 2012 2:10:54 PM
Sorry to come in late to this discussion. I think the issue of whether and to what extent lawyers face time pressure in practice is a red herring. I don't believe that's the reason why time limits are placed on exams (or rather, if that IS the reason, I don't think it's a good one). Time limits are placed on all sorts of exams; history exams, for example. No one calls up a practicing historian and says "Quick! Why did the American colonies revolt against Great Britain? I want an answer in one hour."
The purpose of time limits, as far as I can tell, on law school as well as other exams, is to provide a mechanism for determining how much of the course content the student has internalized. Pretty much everything in a law school class is basic information that a lawyer practicing in that field should have as background, that they should not *have* to look up -- the purpose of the time limit is to test whether the student understands the conceptual framework of a sample of the course material without performing basic research (even if the test is open-book, the time limit prevents reading for concepts as opposed to looking up precise wording or refreshing recollection).
I still think it's a crude device though, in law and elsewhere, for the reasons people have identified, and I don't have a strong view as to whether take-homes of various lengths are a better or worse testing method. I can report I haven't noticed much difference in my distributions.
Posted by: Bruce Boyden | May 15, 2012 2:23:35 PM
I was in a law program at a top tier school and there was both rampant and blatant cheating on take home exams. Some of it seems resultant of the entitlement attitude of individuals these days where to have actual consequences on the student the school should provide proof of the cheating which is usually impossible. One girl was investigated twice in three months by the Dean's office with a number of direct witnesses and nothing was done - she graduated with no report to her bar. A librarian told me of what she thought was an older one L that would frequently come in for assistance with his research class. A few weeks into helping the man, she realized that he was not a student, but a parent of a student doing the student's work because the student was overwhelmed. Due to cheating concerns, I am aware of one professor that gives unlimited time for in class exams, he orders pizza for students if they are still working at dinner time (and has offered to buy breakfast, but none have made it that long). Take home exams are certainly more reflective of the work that law students will do in their careers, but for me, the honor of the degree suffers from the high level of cheating that students engage in with take home exams.
In my experiences with allowing for extra time for disabilities, the rule is never clear and cannot actually replicate fairness because it is impossible to come up with a certain amount of time to add on to a student's exam to make it "fair"; additionally, it doesn't replicate real world practice. In my JD studies one student received extra time because English was not his first language. Another student whose first language was not English did not receive extra time. The student that did not receive extra time was Eastern European, whereas the one that did, was Asian - both of the students had done their undergrad in the U.S. (Stanford and Cornell), but it was explained that the Eastern European student appeared to have better English skills. Another school allows for time and a half for students that did their first degree in another country; the girl from Canada got time and a half.
If we are ranking our students against one another and sending them out into the workforce with the same certification, they should be judged on the same circumstances. I believe that to give take home exams and to allow extra time to some students and then to rank those students against one another takes away from the equality we seek to teach in law programs.
Posted by: Anon | May 15, 2012 5:44:34 PM
SHG, I strongly disagree with your statement that people with physical disabilities should be more readily welcomed into the profession since, because their disabilities are "flagrant," any potential clients will knowingly assume the risk of having a disabled representative. Some physical disabilities are visible, and some aren't. But this is totally irrelevant, because a client has no need to know of her lawyer's disability (physical or otherwise) unless the disability will somehow have an impact on the representation.
It sounds like you are arguing that blind practitioners (including, I hope, legally blind practitioners whose blindness isn't necessarily "flagrant") should be allowed to practice, while someone who is, say, severely dyslexic, and who uses precisely the same assistive technology to read documents, shouldn't be licensed to practice law or possibly even allowed to attend law school.
I also wanted to respond to your statement that "law schools need to vet their student populations for those, whether disabled or just not intellectually suited, who should not be lawyers[.]"
I don't agree that a disabled student who has demonstrated she is capable of successfully completing law school should be prohibited from attending solely because she will be too disabled to practice law once she graduates.
And note that impairments are not necessarily fixed, static things, and adaptive and assistive technology is changing rapidly. The determination of whether a student will be able to work as a lawyer given her (possibly fluctuating) impairment(s), and given available assistive technology and workplace accommodations, is not necessarily a straightforward determination that can (or should) be added to a law school's "vetting" process for applicants.
That said, for intentionally time-pressured exams, it really is very difficult to perfectly calibrate extra time accommodations. Some disabled students may get too much time, but others will get too little. The results can end up feeling arbitrary and unfair to everyone.
I concede that it's not a perfect process, and perhaps there is more that can be done to make things better and more fair for everyone (including the disabled students--I feel like I once spent more time securing accommodations for the bar than I did studying for it). But if you are correct that our primary concern should be for our students’ future clients, then we shouldn’t be keeping any group of talented people out of the profession.
Posted by: Melissa | May 15, 2012 6:42:41 PM
What is most disturbing about your comment isn't the reflection of the culture of entitlement that sadly infects so many young people as well as the politically correct within the Academy, but that you rationalize deceiving clients for your own benefit.
If students aren't capable of performing the function of a lawyer, they aren't "talented" except perhaps in their own mind. But if students become lawyers who deliberately conceal their inability to perform the function of lawyer, they're not only intellectually incompetent, but ethically as well.
Posted by: shg | May 16, 2012 6:22:43 AM
Great discussion so far, but I have a question for those professors who have given an 8+ hour exam; how (if at all) do you modify the "traditional" issue-spotter exam question? If the point of the issue-spotter is to test the breadth of the information conveyed during the semester--that is, to reward students who have prepared by studying the book instead of the chapter, so to speak--does the longer exam time flatten the curve (e.g., by giving more people a chance to look up information that, in a shorter exam, they would have had to have at their fingertips)? If it does, do you compensate by substituting depth for breadth or, as has been suggested, by requiring a better knowledge of the relative importance of different issues within the issue spotter? Does anyone supplement longer essays with short responses or the like?
Posted by: Seth S. | May 16, 2012 10:28:22 AM
Suzanna's approach actually turns the common approach (that I and several of my colleagues use) on its head--giving the "objective" stuff during the semester and saving the writing until the end. That certainly alleviates the time-consumption problem of grading mid-semester.
Posted by: Howard Wasserman | May 16, 2012 11:09:11 AM
@Seth S: In my 1L course, I given a 8-hour take-home exam. It is designed to take 4-hours or so and includes a strict word limit to encourage prioritizing and editing. The questions include an issue-spotter for breadth, 1-2 short answer essays for discrete and especially important issues, and 1 policy or theory question for depth of insight. (For those interested, I use Examsoft for the take-home limited open book -- essentially barring the internet -- which functions to impede the ability to copy or cut-and-paste statements without understanding them and to "research" beyond the closed universe of exam facts when issues are informed by current cases.)
In an upper-level course, I given an in-class midterm exam that is a fairly typical time-limited issue spotter and a final paper (assigned after the midterm and due on the last day of exams). Rather striving for balance within a single instrument, I figure the dual assessment accounts for different skills and strengths that (1) suit the subject matter (e.g. the midterm covers basic concepts and statutory knowledge whereas the paper addresses unresolved legal and policy issues), and (2) matter to the legal profession.
I've liked the results, even if it requires more grading for me and some justification when students resist nonconformance.
Posted by: M | May 16, 2012 2:44:34 PM
SHG, ouch. I sincerely hope you weren’t accusing me personally of deceiving my clients for my own benefit, but that’s how I read your comment. In any event, I absolutely haven’t. Even under the odd set of standards you’ve articulated here (which I happen to disagree with), I’m in the clear.
Not applicable to me, but I’m honestly curious: if a legally (but not "flagrantly," to use your term) blind person and a severely dyslexic person take their law school exams and their bar exams with the exact same accommodations, and use the same exact same assistive technology in their practices, what principled basis do you have for concluding that one, but not the other, is "incompetent" to practice law?
And if an associate's use of adaptive software to generate documents doesn't impair his ability to represent clients in any way, then how does the fact that he required bar exam accommodations (to enable him to take the examination using the software), along with the fact that his disability isn't one you consider "flagrant," establish an ethical obligation that he make some disclosure to his clients? Which ethical rule, exactly, is he violating if he fails to make the disclosure? How are his clients being deceived?
I also don't understand your claim there is an unacceptable risk of harm to clients whenever, due to some non-obvious impairment, the holder of a generalist license to practice law can't perform every type of attorney job. Is *anyone* always physically able to perform every essential function of every possible attorney position? And if, for example, you were to acquire an invisible impairment that left you unable to perform some types of legal work that aren't even part of your practice, do you think your license to practice should be suspended until and unless you fully recover? After all, during the period of your disability, you wouldn't possess the criteria you argue ought to be required for holding a generalist license to practice law.
I think you have been conflating "capable of taking the bar exam without accommodations" with "capable of being a competent lawyer." These are not the same thing, for all the reasons others have explained above. Otherwise, many still-productive and talented attorneys would become technically “incompetent” as they age and develop limitations in their abilities to sit for long hours and type or write out long, multi-day tests, using standard font sizes, etc.
Posted by: Melissa | May 16, 2012 3:56:43 PM
I wish I saw this post a few days earlier.
My 1L year, I had a temporary psychiatric disability. The school was happy to offer me additional time on the exams, and the dean told me that there was nothing else that they could do - extra time was it. I took it the extra time, and I finished most of my exams in the standard time allotted.
My 2L year, I took disability law. The professor gave us an 8-hour take home with about 2-3 hours worth of exam material. She told us that she was giving everybody additional time in the interests of universal accessibility. I finished it in the allotted time. The other three students in the class, gunners all, worked to the very end of the time limit, without even taking significant meal breaks. You can probably guess the results.
shg - a significant proportion of law school graduates go on to become worthless lawyers, and they do so for innumerable reasons. Why single out the disabled? Case very much in point a friend of mine, recently returned from a gnarly tour in Afghanistan, with serious PTSD and associated ADHD that made it harder for him to concentrate in other people. He's court-certified and has been representing clients through the local public defender's office for the last year. He takes great care of his clients and is one of the best oral advocates in the class - in fact, he just yesterday gave the graduation speech. He might not have made it through his first year without these modest accommodations, but I'd rather have him represent me in court than anybody in the top quartile of our class.
Posted by: Zach | May 20, 2012 7:10:01 PM