« Distinguishing Article IV “Privileges and Immunities” from Section One “Privileges or Immunities” | Main | Goodbye and Thanks »

Thursday, October 01, 2009

How Can We Demonstrate That Our Students Are Learning?


For those of you who don’t know me, I’m Rick Bales. I served as interim dean at my institution (NKU-Chase) a few years ago, have a long-term interest in becoming a dean, and am chairing this year our self-study committee.  Consequently, I’ve had law school governance issues on my mind a lot recently.  Over the next several weeks here on PrawfsBlawg, I hope not only to share a few of my own observations, but also to solicit comments from readers on a variety of law school governance issues.  My posts will be deliberately provocative, and if you (dis)agree with what I say, I hope you will respond.  I’d very much like to benefit from the collective wisdom of the PrawfsBlawg community.

An issue that’s hot right now in higher education generally, and becoming hot in legal education, is outcomes assessment.  How do we know that students are learning anything from our courses?  Single, end-of-semester examinations and bar examinations are poor determinants of student learning.  They both are summative rather than formative, meaning that students do not find out until it is too late whether they are learning the material at an adequate level (contrast weekly quizzes – if a student regularly performs poorly on quizzes, the student knows she needs to step up her game).  They both are snapshots: they demonstrate what a student knows at a single point of time rather than what is retained over time.  For both, there is very little correlation between what is tested, how it is tested, and real-life law practice.

Publicly funded undergraduate institutions are under increasing pressure from states and accreditation authorities to justify their subsidies by demonstrating that student learning is occurring.  Primary and secondary education has seen this for years in the form of standardized testing.  Outcomes assessment is coming soon to law schools.  How should we respond?

Rick Bales

Posted by Workplace Prof on October 1, 2009 at 08:45 AM | Permalink


TrackBack URL for this entry:

Listed below are links to weblogs that reference How Can We Demonstrate That Our Students Are Learning?:


When I taught estate planning to non-lawyer graduate students, we did benchmark exams before and final exams after to measure student learning. The results were reported in a "spider web" chart with average scores before and after for different subject matter sub-topics marked on lines radiating from a common center. They were visually pwoerful, but, that really only rewards professors with the least familiar topics.

More importantly, this actually measures student learning, and by implication, a professor's teaching excellence, which is not what we want law school exams to do. The professors mostly have tenure, so why bother measuring their performance? Student learning is irrelevant as well, as I explain below.

I also disagree that the measurement format is much different that real life for lawyers. It may be bad pedagogy, but it is very much like a lot of trial and transactional work.

In legal practice, you spend large amounts of time preparing for a single make or break evaluation, conducted in a high pressure, high stakes environment, with little or no feedback until the entire task is over. Law school exams are for lawyers what multi-day residency shifts are for doctors; tests of an ability to perform under pressure.

Anyway, as I noted before, measuring "student learning" isn't what one seeks to measure on law school exams. Law firms and others who use law school grades want to know if the students are able once the course is over. They don't really care if the student knew it all in advance (e.g. in the case of a student who took undergraduate constitutional law and then took constitutional law again in law school), or learned it while in law school. Student learning is irrelevant. Only student achievement matters to the people who use law school grades. They are a sorting mechanism and a means to maintain minimum standards.

The only law school and bar task that bears a real close relationship to what trial lawyers do is the character and fitness review, which requires bureaucratic competence in the form of assembling large quantities of documents and answering highly formalistic questions according to detailed rules. Most of the time, in all areas of practice, lawyers are more akin to "superbureaucrats" than they are to academics, scholars or intellectuals.

But, this doesn't mean that we should teach people to be bureaucrats in law school. Like a lot of things that law school doesn't teach, it is ill suited to classroom instruction.

Posted by: ohwilleke | Oct 1, 2009 6:10:16 PM

Some may wish to explore how "clicker" technology can be used in the law-school classroom.

Clickers, such as the iClicker device , are a form of active-learning, classroom-response technology that appear to spur student participation and increase both comprehension and retention. (E.g. ; ).

Clickers also help instructors better gauge pace (when students have grasped a certain concept and are ready to move on and when instead it's time to slow down and review or clarify). Finally, the clickers provide instant, electronic, organized assessment data. (Well, maybe not formal "assessment" data but some sense of what percentages of students are and are not learning certain things at various points in a class session or semester).

Posted by: Derek Kiernan-Johnson | Oct 1, 2009 11:20:19 AM

I wonder what you (and others) think of problem-based learning as an early response to the ABA's and others' calls for more frequent assessment in law school. Are weekly (or daily) problems, worked in class, a sufficient response to the need for formative assessment? Does it matter that these exercises are not formally "graded"? Those of us who use this method offer our students constant feedback, at least if the students do the problems ahead of time and compare their analysis and responses to ours. My sense is that heavy problem-based teaching (as opposed to some variant of Socratic dialogue and case analysis) already responds adequately to calls for more formative assessment. No?

Posted by: Jason Kilborn | Oct 1, 2009 10:49:16 AM

Rick, I'm glad you posted on this -- one other data point to add is that it may be coming to law schools sooner rather than later, as the ABA is moving forward with proposed changes to their standards to make us pay closer attention to this.

Posted by: Jason | Oct 1, 2009 9:48:21 AM

The comments to this entry are closed.