« A Texas bill that is both stupid and unconstitutional | Main | "The Theocracy Brief"? »

Monday, March 23, 2015

Talk-show-do-test

A few posts ago, I discussed one of the teaching models that the Army uses: crawl, walk, run.  Within that model, at each level, the Army uses another model: talk-show-do-test.  The basic idea is that you talk to the students about the skill (these can be manual skills or thinking skills); you then show them how to do what it is that you want them to do; you then have them do it; and then you test them on it.

This year, I am teaching a "foundational" course (criminal law) for the first time.  As I thought through how I wanted to evaluate my students using the talk-show-do-test model, I was struck by the disconnect between the skills that we tend to focus on in foundational classes and the skills that we actually test.  

In the foundational courses, many of us use some form of the case method to teach the skill of argument deconstruction.  The students break the appellate argument into its pieces and find the blackletter, and we explore the left and right limits of that blackletter when we modify the facts a little bit.  After students have done this enough, the effect is as Kingsfield put it, "You come into here with a head full of mush and leave thinking like a lawyer."

If we look at Bloom's revised taxonomy, we see that the case method focuses on higher order thinking skills.  We are asking the students to analyze: can they "compare, contrast, criticize, differentiate, discriminate, distinguish, examine, experiment, question, test".  The skill of outlining (or organizing) also falls in here (finding coherence, integrating, outline, parsing, structuring).   And when we question the public policy reasons for the rules, we are asking the students to evaluate: "appraise, argue, defend, judge, select, support, value, evaluate". 

In the foundational courses, we should be working on these skills in class and I think we do a good job with the "talk-show-do" on these skills.  (Notice here that Kingsfield "shows," after first demoralizing the story's hero).  One of the reasons why the 1L year may be so hard is that as undergrads, our students spent most of their time on the lower order thinking skills (remember, understand).  On the first day of law school, we jump them several steps up the pyramid.  

We "talk-show-do," but I'm not sure we "test" these skills very well.  We tend to use cold-call roulette to hold each student accountable for case deconstruction, but that isn't a very accurate way to measure whether the student has mastered the skill.  Nowadays, students can (and do) download case briefs (there is one online for every case in pretty much every major casebook) and they can use those to survive the moments when they lose the roulette game.  We may be measuring their Google skills and not their ability to take a case apart. 

Further, if they don't do well in that in-class moment, the consequences generally aren't that significant (compare the weight of in-class participation to the weight of the final exam). 

Instead, in final exams, we tend to drop back down Bloom's pyramid to see whether the students can "remember" the blackletter rules and "apply" those blackletter rules to new situations. 

For those skills, we "test," but we don't "talk-show-do."  We don't use our class time to teach the students the skills of issue-spotting, rule application, and how to write up that application (itself a skill).  Then we get frustrated when students "remember" blackletter rules that we never covered in class (and so they must have gotten from a commercial outline) or can't write a coherent answer. 

I don't think I can blame the students for going to commercial outlines to get blackletter rules.  They know that "remembering" is heavily-tested and they want to remember as much as possible.  And I don't think I can blame them for not living up to my standards for exam writing if I don't engage in "talk-show-do" on that skill.

If that set of skills (issue spotting, rule application, and write-up) is important, and I think it is, then maybe we should invest some time into the "talk-show-do."  What I have in mind is low-stakes, in-class or out-of-class problem solving where students get feedback on whether they have mastered that skill -- well before an all-or-nothing final exam.

And if those higher order thinking skills are important, (of course they are), then we may need to come up with ways to test those skills. 

I decided to sample those higher order thinking skills by having my students turn in a case brief.  I looked behind the curtain and I was surprised by what I saw.  They still needed significant work on this skill. 

The case brief isn't the actual goal -- the case brief is just one format for memorializing the precise thinking that goes into case deconstruction.  The repetition of that thought process through structured practice is what causes the brain to reformat.  At some point the thought process becomes internalized, but to get there, the students have to repeat that process, correctly, over and over.  My sense is that they were doing it over and over, but not correctly.

I decided that I would have them "do" case briefs, about one per week, for low-stakes.  And I started "showing" them examples of my own case briefs for the cases that we covered in class but which I did not assign.  As they continue to practice, they are getting better. 

So that covers "talk-show-do," but I still don't have a "test."  I'm thinking that next year I may include this skill as a separate, take-home part of the final, where the stakes are higher.  One of my colleagues, Louis Schulze, thought about giving the students a new case, where they would have to deconstruct it to get the relevant rule, and then have them apply that to a new set of facts, all in an exam setting. 

Any thoughts or suggestions?

Posted by Eric Carpenter on March 23, 2015 at 06:33 PM in Teaching Law | Permalink

Comments

Thanks Eric i enjoyed that. I left school just as the internet came on the scene. I wouldn't recognise a classroom now and certainly wouldn't want to teach in one. I think the students would eat me alive. Maximum respect to those that teach and deal with unruly kids and their unreasonable parents.

Posted by: Sydney | Apr 4, 2015 7:47:31 PM

The comments to this entry are closed.