« The Habeas Scorecard Doesn't Matter--Except When It Does? | Main | Dean Searches and Head Hunters »
Tuesday, September 21, 2010
Student Evaluation Forms Have Too Many Questions: Strongly Agree, Agree, Neutral, Disagree, or Strongly Disagree?
In a previous post, I cited to anecdotal data showing that the percentage of students completing student evaluations has sharply decreased as schools have switched from paper and pencil evaluations to their online counterparts. The post also noted that one of the main reasons why schools have switched to online evaluations is because they are so much more efficient than older evaluation systems in which administrators had to compile results by hand or at least combine results after running evaluations through Scantron machines. But have we taken advantage of this increased efficiency by unduly increasing the number of questions asked on student evaluations, and has this contributed to the decrease in the percentage of students completing student evaluations?
At my school, student evaluation forms currently have 24 questions. This is markedly more than the number of questions on forms I used to fill out. To the best of my memory, most forms had between 5-10 questions when I was a student. I am on the Student Affairs Committee at my school, and we just put forward to our faculty a modified student evaluation form that only asks 5 questions. The theory is that with a reduced number of questions, we will increase the percentage of students completing evaluations. But will it work, or will students who don't feel like answering 24 questions feel the same about answering 5 questions (or do they not fill out forms based upon other concerns such as concerns about anonymity)?
Of course, this begs the question of which is better: 50% of students filling out a 24 question form or 80% of students filling out a 5 question form. Certainly, if we're using evaluations for tenure and promotion decisions, we would prefer the latter so that we can have statistical significance. But, if professors care more about student feedback on a variety of teaching criteria (24 at my school) than about the fact that 48 rather than 30 students are giving that feedback (on only 5 criteria), we would prefer the former. So, what do you think? How many questions are on your student evaluation forms? Has that number increased in recent years? Would a decrease in the number of questions significantly increase the percentage of students completing student evaluations? And would such a change be desirable? You can answer by responding to the following poll and/or leaving a comment.
-Colin Miller
Posted by Evidence ProfBlogger on September 21, 2010 at 10:32 AM in Teaching Law | Permalink
TrackBack
TrackBack URL for this entry:
https://www.typepad.com/services/trackback/6a00d8341c6a7953ef0134878c4c5e970c
Listed below are links to weblogs that reference Student Evaluation Forms Have Too Many Questions: Strongly Agree, Agree, Neutral, Disagree, or Strongly Disagree?:
Comments
The only book on this I still have on my shelf is Raoul A. Arreola, Developing a Comprehensive Faculty Evaluation System: A Guide to Designing, Building, and Operating Large-Scale Faculty Evaluation Systems (Anker Pub. Co., 3d ed. 2007). It was recommended to the Chase faculty by Prof. Megan Ballard at Gonzaga Law, who conducted a workshop to get our faculty to really think about how we should change our current five-question form. The questions asked should correlate to the mission of the school. Prof. Ballard was incredibly helpful to us. There are companies that specialize in creating statistically relevant evals, but they can charge quite a bit. Good luck!
Posted by: Jen Kreder | Sep 23, 2010 2:09:58 PM
When I was a student, the chances of me responding to an evaluation decreased exponentially with as the number of questions increased. And yes, I mean exponentially.
Here's a question: How much utility do professors gain from the multiple choice parts of an eval form? I.E., questions like "On a scale of 1 to 5, rate the professor on (1) How knowledgeable he/she was about the subject; (2) How prepared he/she was for class . . . etc, etc."
I always felt those didn't really provide a lot of real feedback. Add to that the tendency I noticed between my friends and I to give professors the benefit of the doubt on those portions and I'm not sure what our evals really told professors.
On the other hand, whenever I really wanted to provide an eval for a professor, either because I wanted to compliment and say what worked or give feedback on some problems I saw, I always wrote out comments.
In short: How are evals used by professors? Are the multiple choice given much weight? Are the written comments helpful, or too rarely written by students?
Posted by: John Nelson | Sep 22, 2010 7:21:07 AM
Jen, thanks for the comment. If you could post an additional comment with some of those books, I would greatly appreciate it.
Posted by: Colin Miller | Sep 21, 2010 6:34:15 PM
On the other hand, a single question geared toward a certain dimension of teaching also is statistically insignificant. There are some good books out there on designing faculty evaluation forms. Unfortunately, I'm not in my office and can't put my finger on the one on my shelf at the moment. Maybe others can suggest some?
Posted by: Jen Kreder | Sep 21, 2010 6:09:05 PM
Absolutely yes. The key point on evaluations is return rate. Studies have shown that less than a 2/3 return rate is meaningless.
Posted by: Mark | Sep 21, 2010 12:56:18 PM
The comments to this entry are closed.