On Student (and Faculty) Evaluations: Some Good Reading and One Modest Proposal

The Chronicle of Higher Education has published an interesting lineup of pieces on end-of-semester student evaluations, a perennial subject of interest for academics. The "con" side is represented both well and more thoughtfully than usual by Michelle Falkoff, a clinical associate professor of law at Northwestern. The title of her piece (which she may or may not have chosen or approved)--Why We Must Stop Relying on Student Ratings of Teaching--is subtly indicative of that thoughtfulness. It is not a plead for abandoning them, but against relying on them solely or heavily. The main argument often brought out against them is made here, and in another piece: student evaluations tend to treat women and people of color differently and worse. Beyond that, however, they may also display "biases that fall outside traditional categories of discrimination," including "student negativity toward classes they perceive as overly challenging or taxing," that "harm an institution’s ability to use student evaluations to gauge instructors’ effectiveness." These trends have been added to by other negative features as universities move toward online evaluations, which have reduced the number of students filling out the forms and tend to adopt the snark of other online writing.

The "pro" side is also represented in the issue, refreshingly, in a "Defense (Sort Of)" of student evaluations by Kevin Gannon. Gannon writes that student evaluations are "a flawed instrument" at best and a "cudgel used against faculty members" at worst. But he argues that whatever students don't know about what they're evaluating, they are still "experts on what they experienced and learned in a course, and they ought to have a voice." And he too cites studies, which suggest that despite their flaws, student evaluations are still some of the best measures of faculty effectiveness.

My desire here is not to take a side between some reductive version of "pro" and "con," although some professors do have a fairly reductive negative view of student evaluations. One reason for that reluctance is my fairly blindered perspective. Like all professors, I have received nasty and unhelpful evaluations. (I have also, to my shame, had bad semesters in which the evaluations reflected the fact that I did not teach as well as I should have. I try to take those moments to heart, weeding out merely hostile rants but looking for common complaints that suggest areas of improvement and trying to implement them in the next class. What I ought to do every semester, but generally don't, is survey my students at least once early or in the middle of the semester, while there is still time for mid-stream improvements.) But students, so far as I can tell, don't judge me for what I wear (and I often dress unconventionally for class), don't apply irrelevant criteria for evaluation, and don't impose other unreal or uneven expectations or stereotypes on me. If I received such evaluations as a matter of course and knew that the data suggested they were likely to be more hostile because of irrelevant factors, I would not be keen on them either. Since I don't face such barrages, I am inclined to accord greater weight to the complaints of those who do. (I am not a fan, however, of those popular videos of professors reading hostile student evaluations, half in pointed humor and half in anger, just as I'm not a fan of the endless stream of "It's in the syllabus" complaints professors favor on Facebook and elsewhere. Students should be treated with respect, given that they are both a main part of our callings and the source of our livings. Everyone vents and jokes about their jobs, but more dismissive professorial treatments of students are all too common in the private and only semi-private spaces of social media.)

As Gannon argues, though, students still deserve a voice in their educations. If they are not simply "consumers," neither are they inconveniences or adversaries. And there is certainly such a thing as more or less effective instructors. What I admire about both his and Falkoff's pieces is their refusal to throw the baby out with the bathwater by, say, suggesting that we get rid of student evaluations while remaining vague and cursory about proposed "alternate methods of evaluating teaching effectiveness." Falkoff, in particular, rather than simply launching arrows at student evaluations, builds on her extensive experience to offer a host of reforms we might consider. Falkoff believes that "holding instructors to high standards is important, and student feedback is relevant." But she believes that we should treat them as only one piece in a "more holistic strategy in which multiple factors contribute to a more accurate, consistent, and well-rounded assessment." Similarly, Gannon argues that the "best faculty-evaluation systems are multilayered and employ a number of different measures," including "faculty narratives, peer observations, reflective dialogue, and sample teaching materials." 

Neither writer talks much about how we could improve student evaluations themselves. Doubtless there's a literature out there on that subject, and doubtless there are costs and benefits of moving to a better set of questions, including a drop in response rates (although clearly the approach of using online, "press a number between 1 and 5"-type evaluations has not resulted in a great response rate either). We could certainly aim to write better and more specific questions, and encourage detailed and specific responses rather than either numbers alone or general invitations for comments that allow students to rant at will. And rather than simply hand a set of evaluations (or a website address) and a brief and mechanical set of instructions to a student to read, we could do in the evaluation-distribution process to explain their purpose and prompt students to offer more serious responses. (Maybe the job of distributing student evaluations or links to those evaluations, and explaining them, should be given to higher-level staff.)   

On the "holistic" side, I do have one proposal to make. Many complaints about student evaluations note that students may not know as much about teaching and about the goals of a particular class as do seasoned instructors themselves (although, in law, few professors learn by anything other than experience and a marginal amount of mentoring by senior professors who may lack little serious pedagogical knowledge themselves; we are not necessarily much more expert about teaching than our students are). I agree that faculty evaluations of faculty teaching should be a major part of the evaluation process. I would suggest the following:

1) Every tenured faculty member should be obliged to visit an equal, and substantial, number of their colleagues' classes each and every semester--say, ten classes per semester--and offer feedback to those instructors and to the administration about the classes they visit. The list of whom to visit and the dates of those visits should be randomly assigned. Every faculty member, including tenured faculty, should receive at least two or three visits by different faculty members every semester.  

2) Those evaluations should involve more than a cursory visit to the class, and sometimes an incomplete visit at that. Professors should be obliged to read the material for that lesson and the syllabus for the course, and stay for the entire class.

3) Evaluations should be always be written and always be detailed. They should follow a set of rubrics designed in advance, including areas of effectiveness, areas of weakness, concrete suggestions for what should be improved or changed and what should be retained and enhanced, and so on. 

4) As I noted, those evaluation visits should emphatically include visits to tenured as well as untenured professors. Length of tenure is no guarantee of good teaching, it is easy to become complacent, and everyone's teaching can be improved.  

5) The law school administration, either directly or through a faculty committee or both, should be obliged to read, collate, and evaluate all those evaluations--not primarily for purposes of evaluating individual teachers, but for purposes of evaluating how well the faculty as a whole teach, what common flaws (if any) they display, and what the best practices are on the faculty. They should be required to write an annual report for all faculty members setting out this evaluation and set of recommendations about what to do and not to do. They should follow this up with a mandatory, dean-and-faculty-led meeting for all faculty to discuss that report, and especially best and worst practices. 

6) Professors who fail without good cause to visit the requisite number of classes and take their evaluation duties seriously, say by failing to write a report or not making it a serious and detailed report, should face penalties, from public shaming to the withholding of one's paycheck or summer research grant until one has completed one's requirements.

What I like about this proposal is that it is burdensome and widely distributed. Tenured faculty members have a duty to their law school, to their colleagues, to their students, and especially to students and to junior colleagues. It should be taken seriously, not just paid lip service. A few professors who are more willing to engage in service than others, and who thus face a disproportionate burden of service as a result, should not be made to do all the work for their colleagues. This is a collective and indefeasible duty. There are good reasons to worry about student evaluations, especially poorly designed and hastily administered ones. But there is an obligation to provide serious alternatives, to make them good ones, and to treat them as a responsibility of the entire faculty, individually and collectively. Teaching is a or the central part of our job, and we should be obliged to take it seriously, both at an individual level and at a collective and institutional one. And, despite the serious reasons to dislike student evaluations, tenured professors who merely take pot shots at them from the side should be obliged (along with everyone else) to be heavily involved in making sure our students receive the best possible instruction.

I think faculty members who take seriously either their teaching responsibilities or their faculty governance responsibilities, or both, will welcome such a proposal. I should think that faculty members who worry most (and most understandably) about bias in the evaluation process should welcome a system that is more serious and systematic in providing a better means of evaluation alongside (and not simply replacing) student evaluations--which, to be sure, ought to be improved as well and should not be given undue weight, at least without culling them, looking for genuine patterns and problems, and so on. (Student evaluations should also ideally be offered more than once and not simply on the penultimate day of class.) They too should be happy to be a part of the solution, even if it is burdensome, as long as it is universally distributed.

I suspect that it is just possible that a few professors will stamp their feet and complain about having to do a great deal of extra work. (No doubt one or two will find a way to work the phrase "academic freedom" into their diatribes.) But I don't think that, say, 30-60 hours per semester spent on mandatory duties aimed at improving the teaching quality of our institutions is an unreasonable demand on us, given the importance of teaching in general, especially in a professional school, and the fact that teaching and service are both major components of our duties as professors. And it is frankly a good thing to smoke out those professors who enjoy complaining but are less than eager to do something about the things they complain about.

I should add that various universities and law schools may already do some of these things. Some schools, for instance, have post-tenure review, and others may simply take our teaching responsibilities more seriously. I'm happy to hear in the comments about more concrete examples of what schools are already doing. And I'm happy to hear about alternatives, both for improving (rather than eliminating altogether) student evaluations and for improving faculty evaluation of teaching--although I think it is valuable and important for the latter to involve serious and universal duties on the part of the tenured faculty.