Monday, December 18, 2006
Enforcing Word Limits
I'm in the midst of grading, and have run into a problem I had not yet encountered in my (admittedly short) teaching career:
On my Federal Courts exam, I have strict word limits for each question. The students are directed to answer three short-answer questions (from a list of five) and an essay question. The word limits on the short-answer questions vary, but are generally ~200-250 words.
The problem: I just finished reading a student paper that was very good, but that was over the word limit on each question. Way over. Examples: For one short-answer question, with a 250-word limit, the student wrote 495 words. For another question with a 200-word limit, the student wrote 408 words, and for the third question, also with a 200-word limit, the student wrote only 315 words.
The question is how to take this flagrant disregard for the word limits into account. Clearly, some penalty should be imposed, or else it would be unfair to all of the students who abided by the (rather draconian) limits. And it's not like I wasn't clear that there would be word limits, and that those limits would be enforced; I was. So, do I just stop reading at word-limit-plus-one, or do I reduce the overall grade for the answer by some fixed percentage (and if so, what?). I've never been the biggest fan of the former approach, because it's hard to stop reading mid-sentence.
So, my tentative course of action is to grade the answer in its entirety, and then reduce it in proportion to the percentage of excess words. Thus, for the questions for which the student wrote twice as much, halve the grade; for the question for which the student exceeded the word limit by 50%, take 1/3 off the grade for that question.
But I'm curious for others' thoughts. Am I being too harsh? Not harsh enough?
TrackBack URL for this entry:
Listed below are links to weblogs that reference Enforcing Word Limits:
» Word Limits from Crescat Sententia
What should a professor do when students violate a clearly-stated word limit? Assuming the answer is not, "ignore it," what remedy is appropriate? This question (by Steve Vladeck) has produced a surprisingly lively comment thread at Prawfsblawg. I weig... [Read More]
Tracked on Dec 19, 2006 12:47:32 AM
» If I Were a Law Prof from De Novo
Ya ha deedle deedle, bubba bubba deedle deedle dum. A couple of Sententias bloggers are posting about the proper way to deal with readings and exams. Raffi supports David Post's controversial call to make all law school reading full text... [Read More]
Tracked on Dec 19, 2006 3:06:31 AM
I had several professors with strict word limits. One made very clear that every ten words over the limit would result in a one point reduction in the final grade. I found that I had plenty of time to edit and used contractions and other work arounds to get below the limit after shooting a bit over it. Of course, I presume that the practice involves using exam software that keeps track of words. It's nutty and cruel to impose a word limit on hand writers...although what the professor in that case did was count lines. Harder to edit that way.
Posted by: Bart Motes | Dec 25, 2006 2:20:21 PM
"If the reason is "I wanted to encourage the students to think through their answers, rather than write as much as they possibly could in the short time provided." then I would say the "spirit of the law" needs to take precedence. If an answer of 500 words is written in such a way that it can't be shortened to 250 words without severely damaging the content, then the student *has* met the spirit of the law and penalizing her for excess wordage would be effectively penalizing her for "knowing too much"."
That is the problem with the whole issue here.
You've stated that the answer was thought through. As a result, the student may very well have thought they would be ok, since your stated goal was to avoid blather, not to impose a word limit for other purposes.
Unless you can say that the questions were blather because of excess wordage (and you've already set this up to state that you can't) you have a position where you telegraphed to the student that you had a "soft" word limit -- one designed to punish blather rather than one designed to put all students on an even footing.
At the same time you have other students who followed the word limit and who would probably feel wronged if the other student was given a benefit for going over.
(a) restating the limits, the reasons and the penalties clearly for the next exam;
(b) reviewing every other answer and moving all of the people who complied with the limit up by the implied improvement their answers could have gained by going over the word limit.
Which, of course, is impossible to do fairly.
Which means you probably need a consult from your ethics committee about what to do next.
Posted by: Stephen M (Ethesis) | Dec 24, 2006 11:14:33 AM
He violated a rule, but it was a rule regarding the FORMAT of the exam. He didn't sneak in books or cheat-sheets; he didn't ask his buddy for some help; he didn't break into your office and glance at your grading grids.
The analogy would be if you offered the student a choice of answering two out of three questions, and the student answered all three.
The intuitively proper way to deal with such violations is simply to select two of the three questions for grading without reading the answers. That way the format violation does not provide the offending student with any advantages due to his violation. I think most would agree that failing or penalizing a student for answering all three questions would be draconian and silly.
Unless there is an INTENT TO DECEIVE here, i.e. the student thought he could slip larger answers in as though they were under the word limit, I see nothing besides a foolish violation of a format rule, easily remedied.
I also really don't see why we would analogize to the consequences of failing to follow proper format in submitting a brief to a court. I've yet to hear of the lawyer who waited until three hours prior to a filing deadline before becoming familiar with the format of a brief and starting work on it.
Posted by: anon&anon | Dec 22, 2006 5:20:16 PM
Surely, it is implicit in setting a word limit that you are being tested not only on your knowledge and your ability to apply that knowledge, but to do so concisely. The test had at least two rules that the student was aware of, and they ignored one of the rules.
On a level of fairness, I entirely agree with J. Senior ("This is not about being fair to the student who has openly and willingly transgressed the rules as clearly set out by the professor: this is about making sure that the students who did follow the rules are treated fairly"), but on a level of pedagogical responsibilities, I agree with even stronger reason with James: "Reading the filing instructions for any document is a basic professional lawyerly competence; failure to do so is often open-and-shut malpractice. When the rules for document form are clearly stated--as yours were--failure to comply with them often leads to the entire document being ignored." Mark's suggestion ("Have him use the form of motion in the local federal court ... following all local rules for motions. Let him make his case. Decide accordingly") is appealing, though.
Posted by: Simon | Dec 22, 2006 11:09:33 AM
Clearly, the only solution is death.
Uh, or whatever. Very little hinges on the choice of sanction here. I suspect that law students are all sufficiently risk-averse that either "stop at the limit" or "knock points off" will induce near perfect compliant behavior.
Posted by: Paul Gowder | Dec 21, 2006 4:05:31 PM
Zev, I have no doubt that if law professors taught nuclear physics, you would have a James Grimmelmann (professor-wannabe who's never even worked as a nuclear physicist) arguing that the student should get no credit, possibly ending the student's career before it's begun, because that's how it works in the "real" world.
Posted by: anonlawstudent | Dec 21, 2006 11:41:29 AM
It's just personal preference. I try to write an exam that should take 6-8 hours, but give them 24 hours over which to do it, so as not to penalize those students who have significant extracurricular committments (e.g., families) that would put them at a disadvantage...
Steve, for what it's worth I'll come out and agree with the anonymous posters who've dissented to this. I think the 24-hour takehome is inhumane, especially in a competitive environment such as law school. (I've always argued against 24-hour general exams, which is why I have a developed view on this.) It creates (even if only imagined at first) competitive pressure to spend an unhealthy amount of time on the exam, is much more family-unfriendly than asking them to devote a business day, and means no one can be sure they're not the ones getting left as suckers for not doing one more hour of work.
Posted by: Jacob T. Levy | Dec 21, 2006 8:41:48 AM
AGM, so maybe the point is a bit trite, but in physics, it is very common to be off by an order of magnitude. It is the same type of mistake as addition or subtraction (by same type I mean caused by the same process) but the scale of the error is obviously much larger. The professor wants to see that you know that a given physical process is solved by computing a line integral, he or she is not so concerned that you can compute the answer (we have Mathmatica for that). Here too, the essence of the exam question is to discern an understanding of case-law, not how many words are used--although the word usage is also helpful to ascertain how well the student grasped the material. How is a lawyer different than a paralegal? I think the answer illustrates this very problem.
Posted by: Zev | Dec 20, 2006 11:44:47 PM
I'm still for treating the student as you would any other who violated exam rules. The instructions you posted show that they had fair notice. I'd treat them the same as if they'd kept writing after time had been called (at my school I believe that's at least a letter grade reduction) or if they'd completed some other knowing infraction.
Posted by: Jesse | Dec 20, 2006 9:24:02 PM
Why is the word-limit imposed?
This is the heart of the question. It is, as I see it, a "letter of the law" vs. "spirit of the law" issue. As presented, it appears that the word limits are imposed arbitrarily (why 250 words?), and are not related to any specific, *quantifiable* reasons.
If the reason is "I wanted to encourage the students to think through their answers, rather than write as much as they possibly could in the short time provided." then I would say the "spirit of the law" needs to take precedence. If an answer of 500 words is written in such a way that it can't be shortened to 250 words without severely damaging the content, then the student *has* met the spirit of the law and penalizing her for excess wordage would be effectively penalizing her for "knowing too much". In this case, I would say that you read the entire answer, grade it based on its content, and then use your judgment to impose a penalty based on weighing the value of the content against the rationale for the limits.
If the reason for the limit is to instill a sense of urgency for clerical restrictions ("letter of the law"), then anything surpassing the limit--even by a single word or character--is grounds for a complete nullification of the answer.
Personally, I would say that the former approach is more applicable. Many commentors have taken a very hard-line approach to this question, citing a strict "letter of the law" doctrine. However, if--out in the "real world"--the law were so black and white, there would be no need for lawyers--and thus no need for law schools--and such conundrums would be moot.
Posted by: Blaze | Dec 20, 2006 8:29:11 PM
Zev, your nuclear physics example is exactly where you would want to hammer a student. Orders of magnitude are a fundamental concept in the sciences, and the ability to use them appropriately is necessary, in the way that practicing physicists or engineers need to be able to (i.e., to determine if the answer is reasonable). You know, to avoid things like a powerplant meltdown or a runaway reaction during fuel processing. From the sound of a lot of other comments, strict length limits seem to function in precisely the same way, with exceeding a limit equivalent in effect: it nukes your motion or filing or whatever it is that is subject to the length requirement. How is an exam question any different?
The question, sounds like, is what sort of punishment is needed.
Posted by: agm | Dec 20, 2006 7:27:34 PM
I have not read the entire comment string, and for any errors that may generate, I apologize.
Steve, in your own words, the reason for the word limit was, "I wanted to encourage the students to think through their answers, rather than write as much as they possibly could in the short time provided." So the motivation was to generate better answers not a specific type of answer. That being said, there seem to be two questions that must be asked: 1. What did the student violate (beyond the simple word limit)? and 2. How do you make the consequence "fair," i.e. prevent similar violation in the future? If you felt, for instance, that a punishment would not effect future rule-breakers, it is not clear why any punishment would be "fitting." The purpose of an exam is to see mastery of a domain of knowledge. If the student did so, irrespective of his or her peers, a good grade should be awarded. Only when the directions impact the type of answer--you wanted a sonata and the student choreographed a tap dance number--should one grade based on form.
I might argue though, that at a certain point the word count becomes a different type of answer. Considering that most grades are given on a curve, it does seem tricky to grade this one against all the others; apples to oranges of sorts. I do like the suggestions of the student stating the word count of his or her answer and consequences being stated explicitly for future violators. Assuming the answer is quality, give it a B and explain why it could not be figured into the curve in a normal way.
To take issue then with Mr. Grimmelmann's "No Credit" comment, school is not intended to mirror the "real world." Education is set up for students to master ideas not mimic processes. It would be absurd for a student to fail a nuclear physics exam for erring an order of magnitude, just because that same calculation would cause serve harm in reality. Taken in the wrong context, all our errors could prove very harmful. That is to say that the real world consequences should not come to bear when deciding the magnitude of the consequences for this rule-flouting student.
Posted by: Zev | Dec 20, 2006 6:12:12 PM
One Harvard professor does not set word limits, per se, but instead states that he reads the first, say 1000 words carefully, the second 1000 half as carefully as the first 1000, and so on. Basically, if you have something important to say, put it up front, but a ridiculously artificial word limit of some multiple of 10 or 25 will not be imposed.
Posted by: HLSter | Dec 20, 2006 2:23:48 PM
Clearly there should be a punishment. How you decide to dock the grade is your choice.
As a note for the future, having just finished a 24-hour take home law exam, I can tell you that, despite the criticism of people such as "Frankenstein" about students who lose sleep on 24 hours take home exams, I spent at least three hours trying to whittle my answer down to the rather draconian word limit. The result was a waste of my time and likely a waste of my professor's time, because he's going to have to read an answer that lost a lot in coherency. I can't wait to get the comments back that say things like "expand on this more," when I would have been docked points for leaving out anything I put in. I ended up at _exactly_ the word limit. While this post may not show it, I can write fairly concisely. The point is, although you should dock points for this student, make sure next time that your word limits are reasonable. It's particularly difficult to have several short answer word limits, because you can't build off of the same starting points as easily.
Posted by: Anonstudent | Dec 20, 2006 2:21:19 PM
As a 3L student, I find this entire discussion utterly preposterous.
Why in the world are you even thinking about NOT punishing this student? As someone who has always complied with the world limit, it irritates me to no end this cry for mercy in some comments.
This is not about being fair to the student who has openly and willingly transgressed the rules as clearly set out by the professor: this is about making sure that the students who did follow the rules are treated fairly.
Posted by: J. Senior | Dec 20, 2006 11:41:26 AM
Bounce the exam. There was a clear violation of well-communicated rules. That is justice.
For mercy, allow the student to make a motion for it. Have him use the form of motion in the local federal court (or appellate court) following all local rules for motions. Let him make his case. Decide accordingly.
Posted by: Mark | Dec 20, 2006 11:17:24 AM
People who pull all-nighters on 24-hour take-home exams are simply fooling themselves. You're not going to produce a better product just because you're sleep-deprived. I just finished a 48-hour take-home with a 1500-word limit, and no, it didn't take me all 48 hours to write.
As for the overage issue, I think the only fair thing is to stop after the word limit with a little padding, e.g. the end of the sentence. If it's a 250-word limit, the answer provided is 400 words, but the first 150 words were preamble... well, maybe next time they'll leave the rhetorical throat-clearing out.
On exams with tight word limits, I bust my butt to get under those limits. Sometimes I make the right choice about what to take out and sometimes I make the wrong choice. It just seems unfair to students who actually care (maybe "care" is the wrong word here, perhaps "are concerned with") about word limits to read beyond the stated word limit.
Posted by: Frankenstein | Dec 20, 2006 8:04:02 AM
Posted by: anon | Dec 20, 2006 5:54:05 AM
24-hour take-home is most certainly encouragement for an all-nighter, and most certainly worse for folks with families. any student who tells you otherwise is either a) lying, or b) doesnt really care about or has in any case come to terms with his/her grades to the extent that they've accounted for the effect whatever their extracurricular activities will (must) have on them.
Posted by: anon | Dec 19, 2006 10:52:22 PM
While I understand that overstepping a word-limit by 10 or 15 words may be accidental or even merit you to stop reading at the 250 words, going over by 150 words seems lacking in academic integrity. Is there an institutional policy about violating exam rules that may be more applicable?
Posted by: Jesse | Dec 19, 2006 9:22:34 PM
The proportional-grading method seems to me to tend to be more strict than the stopping at word 250, because in my experience each 'unit' of thoughtful response produces diminishing marginal returns, grade-wise. (For example, C grades tend to be easier to get than writing 1/2 of an A response, in my UM experience).
Say, as a simple example, that 20 'good things' earn an A. And, for sake of example, a student writes 40 good things, with each thing evenly distributed word-wise, but the response goes over by 100%. Under the stopping method, s/he still gets an A. But under the proportional method s/he gets a C (or whatever 1/2 of an A is).
Posted by: Joel Smith | Dec 19, 2006 6:42:10 PM
You know, perhaps if exams were posted to an intranet, students could rat on each other if they found long exams.
However, since professors generally keep exams a secret, nobody can tell whose grade is being inflated (because of a “relationship” with the professor) and whose is being “deflated” because he lacks a certain “special something.”
Posted by: S.cotus | Dec 19, 2006 4:20:54 PM
I agree with Ted -- bounce. Your rule was clear that exceeding the word limit was not accepted. I think the absence of an explicit punishment indicates that answers breaking the clear rules will not be accepted. I know it's extreme, but so is the blatant disgregard for your instructions. Tell him/her that in fed courts (and in fed courts class) disregarding rules gets your client (and you) a fat nothing.
Or, if you disagree and go with something more forgiving, is an honor violation to his/her permanent file possible? It is akin to cheating...
Posted by: anon clerk | Dec 19, 2006 3:24:29 PM
It's for a federal courts class. Why not issue a rule to show cause for violating Fed. R. App. Proc. 32? Wait a minute... that means more verbiage from someone who violated a word limit... never mind.
All seriousness aside, I'd have to weigh in on the "stop at the limit" method (although if it's in the middle of a reasonably sized sentence, I might read to the end of the sentence). If nothing else, that will appropriately penalize students who rely on IRAC to the detriment of actually reaching a conclusion in the alloted space. Then one's grading key that puts (say) half the weight on the conclusion really comes into play in a relatively objective way.
Posted by: C.E. Petit | Dec 19, 2006 1:28:04 PM
The essential question is how you know when a student crosses the word limit. Do your students submit their answers electronically? For those of us who only get hard copies, counting words by hand is not a worthwhile endeavor, which students, of course, understand.
Posted by: Kate Litvak | Dec 19, 2006 1:24:28 PM
One approach which a former colleague used, and that I have adopted, is to count backwards from the limit by the amount of the overage and read only that far. Thus, for example, if my limit were 1000 words and a student wrote 1300, I'd read only the first 700 words. I'm not sure I'd be quite this harsh in a severely time-limited exam -- then I'd simply stop reading at the limit -- but for a take-home with generous time allowed the taker, it seems reasonable to police the limits strictly in the interest of fairness to the compliant.
Posted by: Larry Garvin | Dec 19, 2006 12:12:29 PM
I agree that stopping at the word limit is harsher than reading the whole thing and discounting. I think that the harshness is fair in two ways: (1) It properly penalizes any attempt to game the system; (2) People who can't read and follow directions both risk failing the bar and more importantly risk hurting their clients. They deserve low grades.
Posted by: Michael Froomkin | Dec 19, 2006 11:30:55 AM
Stop reading at 250. It's pretty standard.
Mr. Baude, my guess is that the question of which sanction is harsher depends on how the student wrote the answers. Some 500-word answers may have a 250-word preface and finally get to the point later. Others may spell out their reasoning in 250 words and then (unnecessarily in this case) flesh it out more so at the end. If the former, the "stop at 250" sanction will probably result in something close to 0 points. If the latter, then the "stop at 250" sanction won't do much.
Posted by: anonclerk | Dec 19, 2006 11:20:35 AM
I don't have any good advice for you, but if misery loves company, I can tell you what I'm currently pondering. I gave an in-class exam but because the classroom I was assigned only has a couple of electrical outlets, I let the students go to the library or empty classrooms (or wherever) to take the test so they could use their laptops if they wanted. They got 3 hours plus 15 minutes travel time to get their answer back to me in the assigned classroom. One student turned in his exam 40 minutes late. My Associate Dean ruled that this is not an "honors code" violation, so it is up to me alone to deal with. I polled three colleagues who had similar experiences and learned that one dropped the student half a grade, another dropped the student a full grade, and the third awarded an F. Arggh.
Posted by: AnonProf | Dec 19, 2006 10:41:54 AM
I bounce. Steve Vladeck's instructions couldn't be much more explicit: "Please note that there is a word limit on _every_ question on the final examination and that the word limits will be enforced. Part of the exercise, therefore, is to confine your answers to the number of words allotted. You may _NOT_ exceed the word limits."
If Steve doesn't "enforce" the word limit, it's a tremendous unfairness to the other students who wrote their 250-word answer in anticipation of enforcement. Unless Steve's essay-grading methodology is something other than every other law-school professor's of looking for key concepts and adding up the number of key concepts hit, going over word length has real consequences. And as James noted, clerks have bounced briefs for less; the Seventh Circuit has given order to show cause for less. All of the other solutions contradict the instructions by reading "You may not exceed the word limits" with "You may exceed the word limits, and the consequences will be a reduction in your grade." If Steve's instructions were less clear, my answer would be different, but, here, given a curve, and given the choice between punishing the majority of students who followed instructions and punishing the single student who failed to follow instructions, the choice seems obvious to me.
Posted by: Ted | Dec 19, 2006 8:26:10 AM
If you just stop reading at 250, then students can game the system by including key arguments in the first 250 and adding more depth analysis or less relevant arguments afterwards. If the prof doesn't enforce the limit, the student gets an advantage. If the prof just stops reading at 250, he's no worse off than the honest students.
I think Steve's suggestion is more-or-less ideal. It allows you to fully evaluate their answer, but penalizes them in proportion to their transgression.
Posted by: Chris | Dec 19, 2006 7:45:30 AM
Far too harsh.
Posted by: PG | Dec 19, 2006 3:42:09 AM
Posted by: x_o | Dec 19, 2006 3:07:07 AM
"You could, Steve, be Shylock and demand your pound of flesh; you could also be Portia and show some mercy."
Posted by: AnonProf2 | Dec 19, 2006 2:34:15 AM
How well can you enforce your word limit? Most students who I have asked to admit to exceeding word limits by 3 or 4% at least once or twice in the past. Assuming your students are similar to the corrupt, incorrigible 2Ls and 3Ls I enjoy spending my free time with, isn't there an element of unfairness in penalizing the student who went over by 200 words, but not the student who went over by 20?
Posted by: UTexasLawStudent | Dec 19, 2006 2:32:46 AM
Get a life, people! It's just an exam -- not a tryout for our favorite theories of contract/statutory/constitutional/testimentary interpretation. You could, Steve, be Shylock and demand your pound of flesh; you could also be Portia and show some mercy.
But as long as we're tying this poor character's mistake to trendy academic legal theories, may I suggest that we explore shaming punishments? What about calling the offending dude into your office and yelling at him -- perhaps until you draw tears. Maybe have the Dean there too, to increase the in terrorem effect. Or, even better, make him write 500 times on the blackboard, "The punishment for prolixity is particularly unpleasant." Or assign a punitive research paper on the horrors that await lawyers who blow through word and page limits. Have him pin a purple "P" (for "prolix") on his shirt, Prynne-style, perhaps! Why, this could be more fun than you ever imagined.
Now back to grading.
Posted by: AnonProf | Dec 19, 2006 2:06:43 AM
For what it's worth, add one more law school alum to the list of people who assumed the remedy for exceeding the word limit was for the prawf to stop at 250. That was always the remedy my professors used (though they almost always stated it on the face of the exam). I think this is probably what most students would assume, given its widespread imposition by other prawfs.
Given that these are short answer questions, I don't think Steve's remedy would result in an F for the student --- just an award of half the points the student deserved on those particular questions, which are likely not the bulk of the exam (though I haven't seen the exam).
Steve - you have one other option in addition to the two that have been discussed: Have your assistant contact the student/s who went over the word limit, or draft an email for your assistant to send them (so as to keep their identity concealed from you for blind-grading purposes) and ask _them_ which remedy they prefer - the stop reading or % reduction. This at least solves some of the harm of your notice problem by giving them a choice in the matter, admittedly post hoc.
As I said at the outset of this post, however, I think that most students assume the penalty for exceeding the word limit is that the prawf stops reading at word 250.
Posted by: Lindsay | Dec 19, 2006 1:15:58 AM
I'm actually quite intrigued by the quasi-emerging consensus here that the "stop reading at the last allowed word" rule is harsher than the "grade the whole answer and discount it" rule.
We'd obviously have to know more about the answers to be sure, but if Steve were to apply the "discount" rule, it seems almost certain that the student would fail these questions. On the two questions where he writes double the word limit, that's a maximum score of 50%; on the other, a perfect score will get him a D. On the other hand, it's conceivable that his answers are sufficiently pyramidal that the first half of the answer is better than the second half, and again, it would be hard to do worse than an F.
Anyway, as a matter of legal interpretation, I do think a stop-reading rule is probably the most appropriate textually, and the responses in this forum suggest that it would not be an absurd result.
As to the debate about exam policy between Marty and Steve, I can say that scheduled exams are fairly brutal, chiefly because of how they interact with other scheduled exams. The student who can pick the order and spacing of his exams will have a much healthier finals period.
Posted by: William Baude | Dec 19, 2006 12:43:20 AM
Bruce, I think I agree with you that Steve shouldn't actually bounce the exam. But I think he should tell the student (and future students), that exam-bouncing is within the realm of possibility. Law school is a safe space within which one can learn what to do and what not to do as a practicing lawyer. Making this a "sinners in the hands of an angry professor" moment seems like a good compromise between making the lesson stick and scorching the students.
And anonlawstudent, "flaunt" can mean "to disregard" in that some writers (such as Law Student) use it that way and some dictionaries agree. But other dictionaries and usage guides (including Brian Garner) hold the line, and so do a great many readers, including many persnickety lawyers in positions of authority. Complying with various formal usage rules is a good thing to do as a practicing lawyer, and being (anonymously) corrected in a blog comment thread seems a better way to learn this distinction than many others I can imagine.
Posted by: James Grimmelmann | Dec 19, 2006 12:02:50 AM
The engineer in me wants you to read the whole answer, score it as though it hadn't exceeded the limit, and then mulitply the number of points you would have awarded by 1/(3*((w-l)/l)^(1/2)+1), where w is the number words actually in the answer and l is the imposed word limit. Note: you can muck with the 1/2 and the 3 to adjust the steepness of the initial drop and the depth, respectively. The above penalty factor is asymptotic to zero, so there's no corner cases to worry about. In the form given above, I think it's sufficiently steep and deep to really hammer anyone that went over while still allowing for some credit even on a 10k word answer (the factor is ~0.4 at 300 words and is just below 0.1 at 10k words for l=250). Besides all that, there's just enough math to scare the crap out of any future inummerate law students, _and_ there's enough teeth to guarantee the numerate ones know you mean business!
I say all this after having finished grading (quite generously, if I do say so myself) the semester project for my Introduction to Scientific Computing course, so maybe I'm biased by the desire to inflict a little more punishement those that don't follow directions or come to class!
Posted by: billb | Dec 18, 2006 11:15:32 PM
Those who take the position that a word limit of 250 necessarily implies a penalty of words 251 and beyond not being read probably also think that expectation damages are inherently the remedy for breach of contract and exclusion of evidence the inherent remedy for an illegal search. How about reading and grading only the last 250 words? The middle 250 words? Whichever 250 words would result in the lowest score? Good thing this wasn't an exam in jurisprudence.
Posted by: A Legal Realist | Dec 18, 2006 11:13:06 PM
"Moreover, if the only "penalty" is that the excess won't be read, there is effectively no penalty."
This is only true if the student has managed to construct an answer that, if cut off at the word limit, is just as good as it would be had he not gone over the limit. In all likelihood, that is not the case.
"The student will go on writing as much as he/she wants, knowing that at worst he/she will be in as good a position as if he/she had followed the rules."
Again, the purpose of enforcement should not be to punish the student for having transgressed. This is not a crime of epic proportions. A student has failed to follow instructions on a law school exam. The word limit should be enforced to prevent one student from having an unfair advantage. That is accomplished by not reading beyond the limit. Anything significantly harsher is just the professor being a self-important, vindictive jerk.
Posted by: anonlawstudent | Dec 18, 2006 10:59:39 PM
My suggestion would be for a harsh penalty. Having never graded exams, but graded assignments for students, if a student went over the limit we tossed the paper. There was some leeway for clerical errors (we had soft copies so we could do word counts) and if someone was a couple words over that was one thing, but something like what you described happened at least a handful of times, and each time the paper would get no credit. Personally I think it's harsh but the only fair thing for the remaining students who listened to the rules.
Posted by: anon | Dec 18, 2006 10:50:07 PM
James, I'm glad you pointed out the "misused" word in law student's post. That correction is clearly central to the discussion at hand. But ironically, you're wrong. Flaunt can mean "to disregard."
I have some harsh comments I'd like to make about your first post, but I'll resist the temptation.
Posted by: anonlawstudent | Dec 18, 2006 10:32:18 PM
James, one problem here is that the exam is not a great simulation of a real-world exercise. For example, in the real world, you're certainly required to follow court rules to the letter, but you typically have days or even weeks to familiarize yourself with the rules in a new court, and to seek advice from the clerk, colleagues, etc. as to how to submit your document. And some rules, such as word or page limits, are so typical that every lawyer should expect them in filing something no matter how little time there is to prepare. But I don't believe word or page limits are similarly ubiquitous for exams. And could Steve's students have looked up his rules in advance? (I sent my exam instructions out by e-mail five days before the exam, because they took up most of a page -- but even that seems a little close for detailed study.)
So I vote against "bouncing" the exam, at least if that's equivalent to an F. I think the only 2 realistic options at this point are stopping at word 250, or some sort of grade reduction. Even if the student somehow missed the instructions (were they on a separate page?), I think you have to impose some penalty to be fair to the others who did not.
Posted by: Bruce Boyden | Dec 18, 2006 10:27:19 PM
Dear Law Student,
You mean "flout," not "flaunt." Steve is flaunting his word limits by posting here about them. Flouting is what you do when you openly disregard a rule.
You do raise, however, a great point. While Steve means his word limits to be enforced by a sanction, some remedies (such as his proposal) actually implement prices. Robert Cooter would say that since Steve can identify socially desireable behavior (keeping within the word limits) but can't easily measure the external costs (stress imposed on everyone else by having, effectively, a longer exam, plus unfairness to those who conformed to the limits), sanctions are preferable to prices. That reasoning would suggest that Steve punish word-limit-flouters with a significant sanction, rather than trying to price out the marginal external costs of writing too long an answer. Further, according to Cooter, he should pay more attention to setting his word limits appropriately than to calibrating the exact punishment.
Posted by: James Grimmelmann | Dec 18, 2006 9:52:54 PM
Thankfully you've decided to exact SOME penalty. The worst thing you could do would be to do nothing because that would be tremendously unfair to students who spent time cutting their answers down to the word limit.
As for what you should do, the proper thing is to stop reading at the word limit AND impose a penalty for violating the rules. This method is both fair to the violator and upholds the integrity of the rules and treats the student like the practioner he will soon be.
That being said, what should be done in the future is to require students to attest to their word total and forbid any words over that amount. Any answer that exceeds the word limit should not be read at all. That's a simple and fair rule and there is not reason why every student could not follow it with ease.
Posted by: PK | Dec 18, 2006 9:45:46 PM
For the future, including the penalty for violating the policy is probably the best approach. As for what that penalty should be, however, and as for what to do to the student in the present circumstance, I say give him/her an F. The rule was there. It was clear. It was broken. Egregiously. I completely agree with Law Student that "notice" is not a problem at all on these facts. We don't want to encourage the "efficient breach"; rather, we want to encourage students to follow the rules just because they are the rules. Of course, actually assessing the F will cause you more grief than you may want to endure. Still, it's probably worth it to the students who complied with the rule to see it enforced harshly against violators. Moreover, if the only "penalty" is that the excess won't be read, there is effectively no penalty. The student will go on writing as much as he/she wants, knowing that at worst he/she will be in as good a position as if he/she had followed the rules.
How about a combined penalty: Don't read beyond the word limit AND deduct the appropriate percentage.
Posted by: Mike Dimino | Dec 18, 2006 9:45:06 PM
It's just personal preference. I try to write an exam that should take 6-8 hours, but give them 24 hours over which to do it, so as not to penalize those students who have significant extracurricular committments (e.g., families) that would put them at a disadvantage... My hope is that the word limits are what keeps the exam closer to 8 hours than to 24, but maybe I'm being naive (wouldn't be the first time).
Posted by: Steve Vladeck | Dec 18, 2006 9:38:04 PM
"I'd feel bad giving someone a _scheduled_ 8-hour take-home."
Why? That's what I do. Is there something troubling about it that I'm ignoring? They knew they'd be tied up from 9:00-6:00 today. What's wrong with that?
Posted by: Marty Lederman | Dec 18, 2006 9:31:40 PM
P.S. For what it's worth, I allowed 30,000 characters (not words), and the students are required to attest to it. I could probably easily drop it to 25,000 or fewer with no significant loss of quality, but I tend to err on the safe side, so that students don't have to spend an hour or two pruning.
Posted by: Marty Lederman | Dec 18, 2006 9:28:28 PM
Marty -- School policy. Although UM allows 8-hour take-homes, those are "scheduled" exams, and I'd feel bad giving someone a _scheduled_ 8-hour take-home, as opposed to an unscheduled 24-hour exam that they can take during any 24-hour period during finals...
Otherwise, I'd do 8... :-)
Posted by: Steve Vladeck | Dec 18, 2006 9:27:20 PM
Steve: Just curious -- why 24 hours and not, say, eight hours (which is what I gave for ConLaw)? Doesn't 24 hours simply encourage all-nighters? And reliance on Westlaw/google/the Web/etc.?
Posted by: Marty Lederman | Dec 18, 2006 9:23:43 PM
On my 24 hour take homes, I have a word limit (which I let the students choose), I require the students to use the word count feature and hand-write and circle the word count on the last page, I tell them orally and in writing on the exam instructions that I stop reading at the assigned number of words, and tell them orally and in writing on the exam instructions that if they greatly exceed the word limit I will use it against them when breaking ties at word cutoffs. Students say that they like word cutoffs and they really like knowing that they are enforced.
Posted by: John Steele | Dec 18, 2006 9:15:34 PM
I'm surprised that nobody has commented on incentives yet. As a law student (who has never exceeded a word limit on an exam), if I knew that a professor enforced his word limit by reading the whole answer and then adjusting my grade downward, I would sometimes flaunt the rule. If writing 300 words instead of 200 could get me from a B to an A- answer, I would probably do it and gladly accept the professor taking 1/6 or something off of my grade. If you want to encourage future compliance with your word limits (at least for students who might read this post), then you either have to stop reading once the word limit is exceeded are have a very harsh downgrade. Above all, the punishment has to do more than merely put the violator in the position he would have been in had he not violated. Otherwise, students won't see a reason to adhere to them.
Also, classes are not liberal democracies. I wouldn't worry about notice of a specific punishment. The violator leaned over an edge with his eyes closed, not knowing whether he was going to fall a few feet or a few hundred feet. He can't claim unfairness if it turns out to be the latter.
Posted by: Law Student | Dec 18, 2006 9:07:10 PM
IGC -- It wasn't my own workload I was worried about (if so, I would've cut out the mid-semester assignments). The exam itself was a 24-hour take-home, and I wanted to encourage the students to think through their answers, rather than write as much as they possibly could in the short time provided. That is, I wanted to disincentivize (sp?) 24 hours of non-stop writing, in favor of thoughtful analysis. My hope was that the word limits would require some attention to content...
Posted by: Steve Vladeck | Dec 18, 2006 9:02:47 PM
You may want to adopt the solution, next year, that the federal courts of appeals actually use: to require a certificate indicating they have done a word count and the text is within the word limit. I can't remember what the penalty is for doing a fraudulent version of that certificate (in practice I was always scrupulously honest about it), but in the academic setting I imagine it would be appropriate to say that filing a false certificate will lead to a failing grade or referral to the academic board for misconduct.
Even without specifying the draconian remedies, I think you'd be surprised at how much the need to aver one was within the word limit would reduce noncompliance.
On a separate note, what made you decide to administer an exam with word limits, especially what seems like fairly tight ones? Was it mostly workload reduction? Or was it that old shibboleth about simulating the "real world" -- you'd be hard pressed to find any form of legal writing that was that short....
Posted by: IGC | Dec 18, 2006 8:57:20 PM
No credit. If the exam were a brief or a motion, many courts would have rejected it at the clerk's office. If it were an application for adjustment of immigration status, the client might be on a plane by now. Reading the filing instructions for any document is a basic professional lawyerly competence; failure to do so is often open-and-shut malpractice. When the rules for document form are clearly stated--as yours were--failure to comply with them often leads to the entire document being ignored.
And once you've pointed out that this is how things work in the real world, reducing the grade in proportion to how far the student went over seems like a perfectly reasonable exercise of mercy.
Posted by: James Grimmelmann | Dec 18, 2006 8:26:55 PM
Insomuch as the form of the penalty has in influence on the nature of the violation, well…What that student did was pretty egregious and out of line, so much so that part of me wonders if he/she was just testing you to see if you'd actually dock points for violation of the limit.
For lack of a better example: if I were to commit robbery knowing that I would be punished, the type of punishment, should I be caught, would have an effect on what kind of robbery I committed. If I were to be put to death for any robbery I certainly wouldn't commit it in the first place. If punishment were to be commensurate with the crime, a fine for stealing bread, jail time for armed grand theft auto, I would be much more likely to steal bread.
As a student, retrospectively, I think it would be a great deal more foreseeable that the teacher would just disregard anything after the limit than reading the whole bit and docking the score based on how far I went over, despite the gracefulness of the second plan.
By the same token you also have a point, you said they would be punished for going over the limit, and that when you set the limit you meant it. It was their fault not to ask what the penalty would be.
Now with the blame shifting I'm almost leaning toward your idea.
Posted by: Ben Snitkoff | Dec 18, 2006 8:17:36 PM
The students who stayed within the word limits probably wrote better answers on the whole. If I were you I'd forget about it.
Posted by: Anthony D'Amato | Dec 18, 2006 8:14:43 PM