It's All in the Numbers: How Can We Increase the Percentage of Students Completing Student Evaluations?

Student evaluations. Some people love them. Some people hate them. Some claim that they are reliable and reasonably valid. See, e.g., Arthur Best, Student Evaluations of Law Teaching Work Well: Strongly Agree, Agree, Neutral, Disagree, Strongly Disagree, 38 SW. L. REV. 1 (2008). Others claim that they are unreliable and invalid. See e.g., Roger W. Reinsch et al., Evidentiary and Constitutional Due Process Constraints on the Uses by Colleges and Universities of Student Evaluations, 32 J.C. & U.L. 75 (2005). Meanwhile, others agree with the idea of student evaluations but believe that law schools must fundamentally change the way that we conduct them. See, e.g., Deborah J. Merritt, Bias, the Brain, and Student Evaluations of Teaching, 82 ST. JOHN'S L. REV. 235 (2008).

Some claim that attractive professors fare better on student evaluations than their less attractive counterparts. See, e.g., Todd C. Riniolo et al., Hot or Not: Do Professors Perceived as Physically Attractive Receive Higher Student Evaluations?, 133 J. Gen. Psychol. 19, 30 (2006). Others claim that male professors fare better than their female colleagues. See, e.g., Susan A. Basow & Nancy T. Silberg, Student Evaluations of College Professors: Are Female and Male Professors Rated Differently?, 79 J. Educ. Psychol. 308 (1987). Meanwhile, others claim that Caucasian professors fare better than their minority counterparts. Kathryn Pourmand Nordick, Essay, A Critical Look at Student Resistance to Non-Traditional Law School Professors, 27 W. New Eng. L. Rev. 173, 188, 191 (2005).

In this post, my intent is neither to praise student evaluations nor to bury them. Instead, I seek to raise a new issue with students evaluations and solicit opinions on how to correct it (or whether it needs correcting). That issue? The decrease in the number of students filling out student evaluations as law schools move from paper to online evaluations.

Student evaluations are actually a pretty new feature of the American education system. Apparently, "[t]eaching evaluations were initiated on many U.S. campuses in the 1960s, as students clamored for more of a say in their education." These evaluations thereafter didn't exactly catch on like wildfire, with "[o]nly about 30 percent of colleges and universities ask[ing] students to evaluate professors in 1973." Instead, the adoption of evaluations was more of a slow burn, with more and more schools deciding to use them to the point where it is now shocking if a school does not have student evaluations.

Initially, these evaluations were strictly a paper and pencil affair, with the first quantum leap being the addition of Scantron forms on which students could now bubble in whether their professor deserved a 1, 2, 3, 4, or 5 for organization, clarity, overall quality, and a host of other measures.  Recently, these traditional forms of evaluations have gone the way of the pager, with schools increasingly creating online student evaluation systems with the goals of saving money and increasing efficiency.

Many schools also thought that this conversion would increase the percentage of students completing student evaluations, but anecdotal evidence suggests that the conversion has had the opposite effect. For instance, at Northeastern, the percentage of students completing evaluations dropped from 80% to 54% after the school converted to online evaluations. See Getting students to make their mark.  I know that at my law school, we have seen a similar dip.

And I think that regardless of your view of the reliability and validity of student evaluations in general, we would all agree that student evaluations become more reliable and more valid as the percentage of students completing them increases. Moreover, I have heard that many colleagues ignore the overall numbers on their student evaluations and simply focus upon student comments. Obviously, if only 50% of students fill out evaluations, such professors have a lot less feedback to use than if 80% of students completed them.

So, what can and/or should schools and individual professors do? Our school has decided to award prizes randomly to students who fill out all of their evaluations, but it hasn't really worked. "Harvard, which moved to online evaluations in 2005, began allowing undergraduates who fill out the surveys to view their grades a few weeks early. As a result, participation rose to 96 percent, compared with 65 percent a few years ago." People at my school have floated the idea of students not being able to take their final exam until they fill out their evaluation or docking the grades of students who do not fill out evaluations. Obviously, there would be many negative consequences to such an action, which is why this remains just an idea. Schools could also go back to paper and pencil, but would they be willing to eat the cost and inefficiency (and would today's students even know how to use pencil and paper j/k)?

So, do any readers have any ideas? Is there anything that your school has done or that you have done as a professor to increase the percentage of your students completing student evaluations? And is the drop in the percentage of students completing student evaluations a problem we need to address or something that we should just accept?  

-Colin Miller