Monday, May 05, 2014
Opening Up the Discussion on Student Evaluation of Teaching
Yesterday I gave you links to sources pointing out some reasons why teaching evaluations--as currently done at many schools--do not always provide accurate or useful data. Thanks for the comments!
Today, I'd like to open a problem solving conversation by relating some information I picked up on my way to getting a Ph.D. in higher education: A) Measurement (assessment) is far harder than it looks; B) It's impossible to measure something unless you first identify what it is and C) There's no point measuring something without an understanding of why you want the information.
There is a vast literature available with ideas about evaluating faculty. Have a look here, here and at these thoughts from AAUP. Our problems in legal academe with misusing student evaluations is that we aren't necessarily asking the right questions of the right people in the right way. Evaluating curriculum is very different from evaluating teaching. I think it's more than reasonable, it's important to involve all stakeholders (students, faculty, administrators, alumni, and the general population of future clients) in decisions about what to teach and how to teach it. I will be writing more later about how medical schools have been doing this over the past 30 years. But its not fair (or even helpful) to confuse assessing how well an individual professor is doing her job with what that job should be.
I think it would be very helpful if every school considered its own individual goals and objectives for classroom instruction. And I'm going to put forward some suggestions below about how that might happen.
But first, in the words of noted legal practice guru Prof. Laurie Zimet of Hastings, lets all remember "we aren't going to solve this problem today."
So what does it mean to decide first what we should be measuring?
Well, lets say a faculty decides that every class teaching a subject on the bar exam be organized according to how it will be tested. In Torts, that would mean a professor who spent more time on intentional torts than negligence wouldn't meet this standard. How could that be measured? What about syllabus review?
Or, maybe a faculty decides that each of our students deserves to have their outside of class questions answered within two business days. Of course we ask the students themselves--but why just by anonymous survey? What about interviewing a small group of students and ask them for examples of times they've asked for help and how the professor responded.
I'm not recommending punitive inspections. What about having a peer visit the class a few times during a semester and then immediately meeting with the professor to discuss what they saw? Maybe not for every class, but what about first year professors, professors up for promotion, or where there has been a problem reported?
Is all of this more work than glancing at a scantron sheet--yes. But deciding what needs to be going on in our classrooms and then finding out if that's happening is very important.
Finally--how we evaluate professors is a key component in what kind of teaching we get back (same is true with students). In other words, we are likely to get back what we measure.
So what do you think? What should we be doing in the classroom--and who should be evaluating it? How can we separate evaluation of teaching with evaluation of curriculum? And where in your institution can you find the resources to review how well what you're measuring reflects what you really want to know?
I have been reading James Popham's book "Transformative Assessment" and found it quite digestable (ie. short), readable and enlightening. I highly recommend it for all law faculty. Link for Amazon - http://www.amazon.com/W.-James-Popham/e/B001HCS1NI/ref=sr_ntt_srch_lnk_1?qid=1399329994&sr=8-1
What do you think of the idea of a quality item bank of multiple choice questions that are created by law faculty for courses and shared by all to be used for formative (not summative) assessment?
Posted by: John Mayer | May 5, 2014 6:49:06 PM
I agree that the question of curriculum is different from the question of teaching. But how many schools define the curriculum for their profs to teach? Maybe my experience is idiosyncratic, but I found the absence of such impositions exceedingly weird when I began teaching. In my first semester, I was told I would be teaching criminal law. But I was pretty much free to teach anything I wanted. There were no requirements about what should be included in that general rubric; no review of my syllabus; no review of my exam; etc. There was a course catalog description, but it's not like anyone checked to make sure the subjects in the description were actually taught.
Posted by: Orin Kerr | May 5, 2014 7:38:57 PM
I agree that the popham book is great and i am a big user of CALI. Thank you
Orin, your experience is probably reflective of the law school norm--although i think accreditation standards increasingly require some review. But that's not content based
In fact I'd like to hear from schools who do it differently.
The million dollar question is whether that is good or bad? and who should decide. None of us wants to be told what to teach, but it could help our students if we at least talked to each-other about it. Especially people who teach different sections of the same course
Posted by: Jennifer Bard | May 7, 2014 12:31:16 PM