« When Citizens Become the Product | Main | Want this job? Move five times in eight years »
Thursday, April 12, 2018
Some Reasons to Randomize Cold Calling
After guest blogging at Prawfs once or twice per year for many years, it's now been almost 2.5 years since my last stint. I'm looking forward to blogging about some of my scholarship since then, but I begin with the evergreen topic of cold calling.
The best way to cold call depends on what you're trying to accomplish. If your goal is to get students to air more accurate or carefully considered answers, you might lean on methods that give students advance warning, such as a panel system where students know in advance when they are eligible to be called on. Panels likely create less anxiety for students in total but do not encourage broad student preparation for class.
To encourage broad student preparation, I will suggest, there are good reasons to select students at random rather than using other semi-arbitrary methods professors tend to use. First, doing so reduces the probability that a professor purposely or inadvertently favors (or disfavors, depending on your view) particular students. Second, even when a professor is as fair as possible, some students may have the mistaken impression that a professor is favoring/disfavoring certain students. A transparent method of random selection can fix the misimpression by making the cold calling process feel less personal. Finally, using traditional approaches to cold calling, a cold-called student is less likely to be called on again in the near future, possibly fighting against the goal of encouraging student preparation.
If you were to randomize cold calling, how should you do it? It's important that the randomization method be quick so as not to distract the class and interfere with the flow of discussion. Here's the method I think I would use: At the beginning of the course (or once the roll is finalized), put the names of all enrollees into rows in a spreadsheet. Then append, say, four more copies of the names, so that if you have a class of 50 students, you'll have 250 names. Associate a random number with each instance of a name (e.g., use Excel's rand() function), and then sort the whole list based on the random number. Finally, number each name from 1-250 and, ideally, use multiple columns to get it all on one page that you distribute to students early in the course. Then, at the beginning of each class, select a single random number from 1-250 (say, using a free app on your smartphone) and write that number on the board. This number represents the place on the list where you will begin cold calling that day and will then proceed sequentially down the list.
Aside from being quick and low hassle on a daily basis, this method has some additional advantages. First, unless you cold call very rapidly, most students will realize once the first number is selected that they are unlikely to be cold called on that particular day. Those students will have already prepared for class anyway (your key goal by hypothesis) and can now enjoy the class with lower levels of anxiety. Second, students who realize that they are likely to be called on will have extra incentive to pay attention and are unlikely to be day dreaming when called on--a moment in class that can be awkward for everyone and detract from the class's energy and focus. Finally, those students who expect to be called on during a particular day will likely use brief pockets of time (e.g., right before class starts, during ten-minute breaks if you have one) to further refine their preparation. Such mini-preparation, though distracting if the whole class does it, can give the few students likely to be called on a chance to better articulate their thoughts and helps with our subsidiary goal of encouraging students to air more accurate and carefully-considered thoughts.
I have misgivings about cold calling and rely on it considerably less than my peers. I offer no general defense of cold calling by any means. But to the extent one uses it at all with the main goal of boosting class preparation, there are good reasons to randomize either across the whole class or within large panels.
Posted by Adam Kolber on April 12, 2018 at 07:43 AM | Permalink
Comments
Thanks, Larry! This is a very interesting approach. I'm certainly sympathetic to your concerns about cold calling. The participation incentives from cold calling may be marginal (depending on class size, I suspect) and the increased anxiety may make cold calling a raw deal.
I would add, too, that when I have focused on volunteering approaches, participation has a positive vibe. But in classes that emphasize cold calling, participation sometimes has a negative vibe such that it suppresses participation that would have occurred in more natural contexts.
Posted by: Adam Kolber | Apr 14, 2018 7:17:06 AM
After years of cold-calling, I have abandoned the practice.
The justification for cold-calling seems to be to encourage students to prepare for class -- especially when random cold-calling is employed -- but the incentive is fairly marginal, especially when students perceive no predictable relationship between their performance when called upon in class and their grades. Even when teachers announce that that some portion of the final grade may be based on class participation, if this is handled without transparency, many students doubt the significance of class participation to final grades. The lack of transparent feedback on their class performance further impinges on the value of this approach. Moreover, cold calling increases students' anxiety, and there is considerable research suggesting that anxiety is harmful to learning.
I have concluded that it is better to offer students positive and transparent incentives to prepare for class and then to participate, and for that reason I award points when students volunteer and make what I conclude is a valuable contribution to class discussion (much like the manner in which class participation is utilized in many business schools). Students who are prepared for class and feel that they can contribute are able to volunteer, and soon after class I notify them if they have earned points. There is of course a degree of subjectivity in determining whether a contribution has been valuable, but I tell students to expect this type of subjectivity -- it is precisely the manner in which their work will be evaluated by supervisors when they enter the profession.
Under this system, when students feel unprepared or unready to speak in class, they need not contribute, but if they remain silent on a regular basis, they will fall behind those students who regularly earn points. I post a number representing the number of points earned semester-to-date by the student with the most points (though I of course do not identify the student) so that everyone knows how many points they are giving up to the curve if they choose not to participate. In this fashion, I use class participation as a form of formative assessment, and offer a predictable incentive to prepare for and speak in class.
Larry Rosenthal
Chapman
Posted by: Larry Rosenthal | Apr 13, 2018 3:52:04 PM
Thanks, Victoria! That's interesting that you can consider professionalism when grading. In most places I've taught, at least for most classes, there's some flexibility in adjusting grades for participation but that might not overlap exactly with professionalism.
Posted by: Adam Kolber | Apr 12, 2018 6:25:47 PM
I use a random number generator to figure out who I am going to call on and then write the student names into my notes. If I have 90 students, then the random number generator gives me a number 1-90 and I look at the roster to see the student associated with that number. I have a primary and a backup for each spot in the class where I plan to call on a student (e.g. 2 students per case).
My main goal is to remove perceptions of gender or race-based bias in my cold-calling decisions.
Even though I teach entirely 2L's and 3L's I have found this method to make it most likely for my students to be prepared for class.
Again the approach may be different based on gender/race/age of Professor. Some students may be adequately awed by certain professors into reading without cold calling, but at my level of seniority and as a woman, I find I need that extra incentive/fear of getting called on to keep them prepared, especially as 2L and 3L students.
Finally 10% of their grade is based on professionalism which includes such things as being prepared for class, being on time, treating each other professionally, etc.
Posted by: Victoria Schwartz | Apr 12, 2018 3:51:56 PM
Thanks, Michael! Your post reminds me that there may be a kind of sweet spot to cold calling methods. If the class is very large, students may calculate a low probability of being cold called on any particular day that can weaken preparation incentives. Panel approaches might have particular advantages in very large classes, and one can still randomize within a panel. And three cheers, by the way, Michael, for methods of promoting broad student engagement!
Posted by: Adam Kolber | Apr 12, 2018 10:48:43 AM
I have randomized for a few years. I use notecards that I shuffle on the first day. It has the advantage that I can ask students to write phonetic spellings of their names if they are hard to pronounce. This has helped immensely.
I used to just coldcall randomly, but found that student preparation did not increase. So, I started giving warning to students for particular difficult cases. And, surprise, they were more prepared. So, now I give a warning to the next four students with the proviso that if someone is gone then things can get out of order (or that I might not cover the cases in order).
My students are more prepared - at least the ones I call on - and class runs more smoothly. The tradeoff may be that others are less prepared so I use other methods to get other students engaged, such as clicker slides, random calling with side questions or for disagreement, etc.
Posted by: Michael Risch | Apr 12, 2018 9:23:37 AM
The comments to this entry are closed.