Friday, May 24, 2013
Trouble with the Curve
Though I was taken by surprise by the extent to which my last post touching on the topic of grading provoked rather strong responses, I am now presenting a proposition that I very much hope and assume will provoke controversy. Here it is: There is no (as in, none whatsoever) pedagogical justification for the traditional law school curve, and it should be abolished.
Here are my problems with the curve -- by which I mean a strict curve requiring x% As, x% A-, etc. all the way down to the lowest grades, and not something like a "target mean grade." First, it corresponds to nothing at all. I'm no statistics expert, but even if there is reason to think that students somehow naturally fall out on such a bell curve if you take a large enough sample, I'm pretty sure there's no way any first-year section is actually large enough or diverse enough in its talents to ensure that the curve will be accurate in every case or even most cases.
Second, the curve hides and fails to discourage poor teaching. We should be trying to bring every student in our class up to a fairly high level (although that is not going to happen, of course, with every student). But whether or not most or all students reach whatever we perceive to be the basic level of competence we are shooting for (call that level of competence a "B"), we have to assign a certain percentage of students grades below that level. And in fact, it's better -- or certainly no worse -- if a certain number of students don't reach that level of competence, because then we can justifiably assign them grades below that level. And when students come to see us wanting some justification of their grades, we really don't need to (and in some cases can't) give them any explanation other than, "you got that grade, not necessarily because you deserved it, but because other people did better than you." It doesn't require us to think about what a "B" really signifies, or whether there is any consistency across courses or years in terms of the grades we give.
Now let me outline what I think might be some traditional justifications: First, curves protect against grade inflation. This is undoubtedly true, but it can be accomplished with the far less arbitrary system, such as a target or maximum mean grade for a course.
Second, I suppose one could argue that grades are inherently arbitrary and correspond to nothing in reality anyway. In other words, the only thing a grade ever meaningfully represents (or perhaps more modestly, is ever meant to represent in the law school context) is one's performance relative to others who happen to be in that same class. But this strikes me as somewhat cynical. I doubt many of us fully accept this view. Maybe employers see it that way to some degree. But if employers' expectations are driving the curve, I would first point out that this is still not a pedagogical justification. What's more, even if this is a reasonable defense of having a curve, it has to be weighed against the unfairness of using an arbitrary curve in the first place--one which does not necessarily correspond even to the differing levels of relative ability among students (i.e., even if you can come up with a relative ranking of exam scores that accurately reflects relative strength, which is what employers most likely really care about, you are still required to draw an arbitrary line between a B+ and a B exam, for example, which doesn't necessarily correspond to a meaningful drop-off in quality).
Finally, the other justifications are .... well, I have no idea. I'm out of them. I'm stumped. Can anyone defend the curve?
TrackBack URL for this entry:
Listed below are links to weblogs that reference Trouble with the Curve:
Jessie, thanks for your thoughtful post. I can think of a couple of things a curve could do. First, let's assume that at some top ten schools, students are really of a uniformly excellent quality and there isn't much difference between them. It's still possible that given the level at which graduates of those institutions are asked to perform, small differences matter. Curves provide an opportunity to send a strong signal about who the strongest and weakest candidates are. When everyone gets a B+, there is no signalling effect. It's possible, however, that if everyone is a B+, and we don't force students into a curve, that would send it's own signal - there is no difference. Pick your law clerk based on biology.
A second advantage of the curve is it prevents professors from extending soft-grading mercies to those who won't really benefit from it. At Florida State, where I teach, there are students who consistently score at the bottom of the class. Because there is a curve, those students are readily identifiable, and we can identify the ones who need some additional guidance in preparing for the bar and the practice of the law. Some of them would stand out without a curve, but I know my own natural tendencies are to avoid handing out low grades when I can. The curve forces me to make some judgment calls that help identify students who need the help.
I could go part way with you, on the target mean grade, if there was a real requirement to give a low grade when it was earned. That's what I did with my small copyright section, which was off the curve this year due to its size. I set the median score with the target mean grade Exams more than two standard deviations above the mean earned an A+. I didn't have any exams less than two standard deviations below the mean, but they would have earned a D. There, you can get the signalling effect for the strong outliers, and perhaps have room to detect more outliers on both ends than the curve could otherwise allow.
Posted by: Jake Linford | May 24, 2013 3:25:32 PM
In a world of competition for scarce resources, life is graded on a curve. The point is illustrated by the old story of the two campers who encounter a bear in the woods. one camper starts putting on running shoes, the other says, "you'll never outrun that bear," to which the first responds, "yes -- but I can outrun YOU."
Whether the resource in question is a job, a favorable judicial decision, or a client's business, the decision about effort required to win that resource depends on the relative merit of the competitors for the resource, not by some imaginary yardstick of objective merit.
If real life is graded on a curve, then I suppose that grading exams on a curve gives students a taste of real life, no?
Posted by: Rick Hills | May 24, 2013 3:37:41 PM
Jessie, I'm unclear about an important part of your argument. When you say that you oppose a "strict" curve, what kind of curve do you think that means? Does that only mean a curve that requires an exact percentage of grades to be in each grading category? Or does that include a curve that includes a range of grades for each category? At GW, for example, we have a range of grades allowed in each category as well as a narrow range of acceptable mean grades. Grades have to satisfy both requirements. Is that a "strict" curve that you oppose?
As for the broader issues you raise, the underlying issue is whether law school grades are designed to measure some Platonic ideal of competence and knowledge or to give employers a signal as to the relative lawyerly abilities among students in a class. I tend to think that they are designed for the latter, which I don't think of as a cynical position. The problem is that each professor tests different things in different ways, using different standards to measure different answers using different criteria. Given that, I think it is mistaken to think that the grades that result measure a Platonic ideal of competence in a field. Rather, the grades represent a student's mastery of a quirky individual professor's own view of the field using the methods and questions that this particular professor opted to use. Successful law students figure it out pretty quickly When you take Torts with Professor X, you're not really getting a grade in Torts; rather, you're getting a grade in "Professor X's Idiosyncratic View of Torts." Indeed, this is a limitation of the grading system: Because exams are graded by the idiosyncratic professors, they can only grade exams using the standards or their own idiosyncratic views.
I don't think that's necessarily a bad thing, as it's preparation for the equally quirky experience of practicing before judges. Judges are quirky and idiosyncratic, too. A good lawyer does what a good law student does: figure out what works in front of the "grader" she has. But it means that the value of grades is primarily a relative one: It shows skill at figuring out that professor's view of the law and expressing it in writing. If a curve is too strict and requires an exact percentage of each kind of grade, then that curve may not allow the professor to express the relative ability of each student accurately. If so, that's a bad thing. But the unfairness is in the curve's failure to express relatively ability accurately rather than the curve's inability to report grades that are accurate in some Platonic sense.
Posted by: Orin Kerr | May 24, 2013 4:06:50 PM
In a multi-section first year course, a curve enforces a necessary degree of equity between sections of required courses that students have been randomly assigned to.
A curve also enforces equity between sections of the same upper-level course, and, arguably, between different upper-level courses.
To the extent that professors are tangibly, or even only psychically, rewarded for large enrollments, or tempted to grade strategically to reduce grading by discouraging enrollments, a curve forestalls these undesirable behaviors.
It may also force a degree of equity over time, although I say that with much less confidence (OTOH, there was this story going around law school when I was a student about a law prof who was getting divorced one year and graded really harshly...).
Posted by: Michael Froomkn | May 24, 2013 4:25:03 PM
One justification for a curve in 1L classes is to guarantee equity among sections. If one torts professor rarely gives A's, but another hands them out freely, the students will rightly complain that their grades are dependent on arbitrary section assignments.
Posted by: Carlton Larson | May 24, 2013 5:27:14 PM
What Carlton said.
Posted by: anon | May 24, 2013 8:27:19 PM
whoops. that last comment was from me (obviously). not "anon", whoever that guy is.
Posted by: dave hoffman | May 24, 2013 8:28:23 PM
My thoughts are based on my years as a hiring partner and head of associate evaluation. We used grades to sort candidates because in most cases there is no better information. Sometimes a student worked at the firm or worked for someone we believe is a good judge of legal talent. That was the best information, but it was unusual. Sometimes you will see a candidate with a string of successes in academic and work environments. That’s unusual because it’s possible only for non-trads. We were typically left with grades issued from institutions that are more interested in ranking students than in developing them.
Still, grades do tell you something. As my law partner used to say, “if an applicant got good grades from a good school, she can’t be both stupid and lazy. Perhaps one or the other, but not both.” And across large numbers grades generally reflect engagement levels, ambition and bar passage. (A few years ago, Berkeley revealed that bar passage by class rank, divided into thirds of the class, was 100%, 94%, and 51%.) For that reason, as a hiring partner, I preferred tight, transparent curves. We realized that the higher we went up the food chain, the more the school would hide grades and blur the distinctions in the bottom 2/3rds of the class rank.
Posted by: John Steele | May 25, 2013 9:33:38 AM
From my standpoint, and assuming a base level range of normal competence among the students, my justification for some aspect of the curve is related to Orin's point. The fact is that I don't always know how difficult the exam is when I write it. Whether or not the grade distribution should be mandated (as is the case in our first year classes at Suffolk), I do assess outstanding performance based on the top scores on any given exam on a relative basis. My three hour exams always have 180 points. The highest I ever recall giving was a 174. The lowest highest score I can recall was about 125. My intuition is that the median high score is somewhere between 135 and 145. I have yet to teach a class in which there isn't a single outstanding student. While the abilities of those outstanding students may vary, my guess is that the variance is less than the variance in test difficulty. In either case, I think the fairest answer is to peg excellence based on the performance on that exam of the best performers.
To answer Howard's question from the other post, we have a required grade distribution for the first year classes. For the upper level classes, there is a required B+ median for classes with 40 or more students (I believe that is the right number). That gives the professor a lot of flexibility because one has satisfied the rule as long as the exam in the middle is a B+, no matter what the grades above or below are. And I always point out to students that in theory both the highest and lowest grades in such a class could also be a B+ (it never is). Below 40 students, grading standards are left to the individual professor, but my casual observation is that many or most still use some kind of distribution. (I use sort of a peloton metaphor from cycling for low grades. If you are at the back of the peloton, but at least in it, you won't get lower than a C. But I reserve the C- an D grades for those outliers who couldn't even stay connected with the peloton.)
Finally, I think separating the assignment of letter grades from actually reading the exams and assigning points is a good idea. Again, to Orin's point, I'm skeptical there's a Platonic "A" or "B". I do know that reading the answers is more often deflating than uplifting. But the numbers I write down as to issues spotted and quality of the discussion of each such issue, it seems to me, lie less than my emotional reactions of disappointment, surprise, self-criticism, futility, and concern over declining admission standards. And, indeed, magically, the numbers do seem to fall into patterns every time that allow me, other than at the very margins, fairly to assign grades.
Posted by: Jeff Lipshaw | May 25, 2013 9:42:46 AM
I also agree with John Steele - that was also my experience when I was hiring and evaluating lawyers. I do think grades give some information about analytical, organizational, and writing skills (and conversely say nothing about common sense, charisma, emotional intelligence, or congeniality, all of which may contribute as much or more than the others to lawyer success). But when it comes to the horsepower it takes to unravel a puzzle, as Sam Mussabini said to Harold Abrahams, the coach can't put in what God left out.
Posted by: Jeff Lipshaw | May 25, 2013 10:18:40 AM
I agree with Jeff's point about separating the assignment of points from the assignment of a letter grade. Like Jeff, I assign some number of points (this semester it was out of 1000), then once I see where everything falls along the numerical range, I assign letter grades. This requires some admittedly arbitrary lines between C+ and B-, but in a large 1L class, the breaks in the numbers typically show themselves.
Posted by: Howard Wasserman | May 25, 2013 10:33:18 AM
This is a very interesting post. In larger classes (i.e. 1L), I've just sort of found that things fall out in a curve. Every year I worry that I'll have to "force" something, but it just falls out that way. Anybody else have this experience?
Posted by: Erik Knutsen | May 29, 2013 10:05:20 AM