« Update: Prawfs-Co-Op Happy Hour, May 30th, 9pm, Marlowe's Denver. | Main | Takaki on "Revolutionary" Thinking »
Sunday, May 31, 2009
Giving students feedback on exams
I am halfway through my grading this term, and, as usual, I feel a certain level of guilt at the inadequacy of the feedback that we typically give to our students on their exams. If the only function of the exams is to help legal employers with their hiring decisions, then, of course, feedback is not terribly important. In my more naive moments, however, I imagine that exam-taking could be more than a painful sorting system. Instead, I imagine that it could be a continuation of the teaching process -- a sort of very intense final class -- in which students and prawfs could think about how the material fits together and have a class discussion about the problem.
For exams to serve that function, however, there has to be a feedback mechanism. Ideally, I'd like to have a "post mortem" class in which the students and I would all meet to go over the exam together. Instead, I write up and distribute a "model answer" with marginalia explaining how the arguments in the answer translate (roughly) into points. I also use old exams as hypothetical problems when teaching the course, so that the gap between preparing for the final and preparing for class can be minimized. I've attached my exam question (Download Hills Spring 2009 ARS Exam (Part 1)) and model answer (Download Model answer (with comments)) from this term's course in Administrative & Regulatory State, a mandatory first-year class here at NYU, to give you an idea of my feedback system. (Feel free to crib from the exam question, by the way, if you like it, keeping in mind that both question and model answer are now in the public domain and accessible by your own students).
It is not a very satisfactory system. Few students come by my office to discuss the exam, and I never know whether my model answer makes much sense or just adds to everyone's confusion. So I am curious whether other prawfs have discovered a better, or at least different, way of integrating the exam/evaluation system into the teaching of the course.
Posted by Rick Hills on May 31, 2009 at 01:16 PM in Teaching Law | Permalink
TrackBack
TrackBack URL for this entry:
https://www.typepad.com/services/trackback/6a00d8341c6a7953ef01156fbe6d78970c
Listed below are links to weblogs that reference Giving students feedback on exams:
Comments
I agree with C.E. Petit's point #1 on having multiple grading opps during the semester. Key here, I think, is acknowledging that profs are not going to voluntarily increase workload/grading.
So what I've done is provide a grading sheet or model answer for a midterm or practice exam, when most if not all will read and learn from it, and not bother doing so (or provide high-scoring answers upon request) for the final, when few will look at it. If the final was eliminated entirely, by doing a midterm and a memo, for example, that'd be OK too.
Posted by: Jason Solomon | Jun 2, 2009 9:55:40 AM
I'd like to suggest a couple of things that this dilemma implies, rather than try to grapple with it immediately:
(1) The absence of feedback is a serious problem in law school, but it seems to me that the better solution is to have multiple grading opportunities during the semester, rather than a grade-determinative final. As the very title of the course suggests, law is a process, not a thing... and, particularly in a "procedural" course like the one described, one simply cannot overemphasize the importance of intermediate steps to getting that final result.
(2) That said, it also seems to me that the ExamSoft software bloody well should be modified to allow students to save a copy of what they actually wrote. Better still, how about an e-mail back to students with a copy of their exam and responses, and any model exam/grading key, sent the day that grades are released? That way, they would have the entire "record" in front of them to both reduce panic and conserve the limited supply of pitchforks and torches available for the p/e/a/s/a/n/t/s/ students to use in "reasoning" with Dr Franken-STEEN. After all, we don't let judges get away with issuing final decisions/orders that just say "denied" or "granted" without some reasoning and references to the record attached!
Posted by: C.E. Petit | Jun 1, 2009 11:27:53 AM
Thanks for the thoughtful comment, Another Student. I always regret wracking anyone's nerves. But here is my rationale for the decision to release the model answer before I released the grades. I have learned from past experience that students do not read the model answer if they receive it AFTER they get their grades. (Last year, I invested a lot of effort in writing up a model answer for the exam, posted it on the course website, and found that very few students downloaded it).
The reason for this post-grade indifference to the model answer is given by Anonymous Student above: The exam-taking process is traumatic and exhausting, so students just want to forget about it. This is an understandable reaction, but it is contrary to my own educational goal, which is to get students to think about and remember the material as long as possible -- with luck, until they make their oral argument before SCOTUS and win accolades for their snappy response to questions about Chevron.
Judging from your comment, my tactic succeeded: Students are now (in your words "scanning the model answer." Of course, they are also cursing my name. But I gave up on being a popular teacher a long time ago: It would be nice to get high fives rather than dirty looks from students in the hallways, but, in the immortal words of Popeye, "I yam what I yam" -- someone whose pedagogical choices (banning laptops, cold-calling a lot, releasing model answers early, giving students a hellishly difficult exam question, etc) are destined to reduce my popularity. So I focus on the only course evaluations on which I have any hope of succeeding -- the exam answers and classroom discussion.
Judging from the answers that I've read so far, my course was a success. I'll just have to remember to look nervously over my shoulder for that mob of angry students when I walk through Washington Square.
Posted by: Rick Hills | Jun 1, 2009 9:47:08 AM
This is the morning of Monday June 1st, and I only have three points I want to discuss in this post:
1. Releasing the feedback *before* we get our grades is incredibly unbelievably nerve-wracking, because all we are doing is scanning through the model answers, seeing issues we didn't mention, and mentally resigning ourselves to the dungeons of Law School Mediocrity. This is especially bad because I don't think NYU's ExamSoft lets us keep a copy of our exam answer.
2. Monday section hypotheticals, plus a no-laptop policy ALONG WITH class notes from TA's were two really really really good policies.
3. Lastly, I second the notion above that you should explicitly encourage students to drop by and discuss the exam. Otherwise, the anti-gunner inhibitions built into all of us after 1L is likely to subconsciously stop students. And even then, there's going to be a lot of people who just want to put the whole course behind them, mostly the people who do only just enough work for class as it is.
Posted by: Another Student | Jun 1, 2009 8:05:53 AM
that's a ridiculously hard prompt for a 3-hour exam, imho.
Posted by: andy | Jun 1, 2009 4:30:09 AM
Thanks, Anonymous, for the feedback on feedback. Yes, I've heard that my exam is a traumatic experience. So far, however, the answers seem pretty good to me.
As for ARS seeming unimportant to 2Ls -- I'm shocked! Every public law course or assignment that you ever have, and most private law ones as well, will require you guys to read statutes and regulations carefully and understand how to make arguments about agencies' and courts' latitude in interpreting those statutes. So ARS will help you make nifty arguments for EVERY SINGLE COURSE THAT YOU WILL TAKE AS A 2L. How many other 1L courses can make a similar boast? But wait: There's more. If you are not asked to write some memo or motion or brief on the meaning of some statute this summer (other than ICWA: I know that you guys have ICWA covered), then I will buy you lunch in the Fall. So ARS IS GUARANTEED TO HELP YOU WITH YOUR SUMMER WRITING ASSIGNMENTS. Again, can you say the same for any other 1L course?
In short, ARS rules the first-year curriculum and will help out mightily in the second year and beyond. So despite the completely natural temptation to reach for the Diazepam after taking my exam to forget all about it, I recommend that everyone review the exam question, the model answer, and the model student answers that I'll be posting on June 8th.
Posted by: Rick Hills | May 31, 2009 11:56:37 PM
Prof Hills-
I don't know if this is exactly what you are looking for, but I have 5 points regarding this post:
1) I found the old exam hypos to be very helpful in the course, as it helped show what the exams were going to look like from day one, and it also showed how the pieces of the puzzle were going to come together (or were coming together) throughout the semester.
2) I also thought your old exam model answers were very helpful in preparing for this years exam. There were some parts that may have been a bit confusing (from my memory of the answers), but overall I thought they were great.
3) It may be that many students don't know that there's an open door policy to come discuss exams afterwards. I feel as though there may be different policies for different professors, and rather than coming by and being rebuffed, students are looking for a more explicit invitation to stop by.
4) It also probably doesn't help that your course is in the Spring. Thus, by the time many students are back in the city to even discuss an exam, it was already so long ago that it may seem pointless to do so.
5) Finally, I think that, at least from the reaction I got from a number of my classmates, the subject matter and exam were so difficult that many students just wanted to put the course in the back of their mind once the exam was over. This also may tie into the fact that a lot of students don't see the importance/necessity of ARS as rising 2L's.
Posted by: Anonymous | May 31, 2009 7:48:22 PM
Most of the answers were between 3,500 and 4,000 words. I'd expect them to cover only a third or so of the points that I discuss in the "model" answer: they've only got three hours for the test. But I wrote out an analysis of all the points, because, as I observed in the model answer, everyone covers a different set of points. I ask exam questions with more possible points that any human could conceivably answer in the time allotted on the theory that different students will have different strengths and interests and can pick where they want to focus their energy.
Posted by: Rick Hills | May 31, 2009 7:38:06 PM
Have you considered handing out these documents as PDFs rather than as .DOCs?
Also, a 9000-word model answer? How long did those poor souls have for this monster? It looks like a fascinating exam, but ouch.
Posted by: James Grimmelmann | May 31, 2009 6:36:35 PM
The comments to this entry are closed.