Monday, February 10, 2014
Skating, Judging, and the Role of Direct Observation
Howard raises an interesting question about how a skating judge’s direct observation of a figure skater’s routine might affect subsequent evaluations of that routine. A study of judges in a different sport -- this time baseball umpires -- may provide part of the answer. The study allowed umpires to observe videotaped pitchers throwing warmup pitches. Some of the pitchers were “control” pitchers, with 70% of pitches down the middle of the strike zone and 30% out of the strike zone. The other pitchers were “wild” pitchers, with the ratio of balls and strikes reversed. The umpires were then asked to call balls and strikes for several dozen more videotaped pitches by the pitcher they had observed, with most of the pitches deliberately being close calls.
The researchers hypothesized that the umpires would call fewer actual strikes correctly for the wild pitchers and fewer actual balls for the control pitchers. That is, they assumed that the umpires would take what they had observed into account and assume that a close call from a wild pitcher was more likely to be a ball and a close call from a control pitcher was more likely to be a strike. In fact, the umpires were influenced in the opposite direction: wild pitchers got more strike calls and control pitchers more balls.
This is only one study, but it suggests that after brief, direct observation, a judge or decision-maker builds a mental image of the athlete’s competence and skill, which in turn establishes expectations of future performance. Pitchers who set the “control” bar high in their warmups were expected to demonstrate similar control in a game situation, and close calls were evidently seen as a failure to perform to expectations. Similarly, pitchers who showed less control in warmups were expected to demonstrate less control in a game situation, and close calls were evidently seen as “close enough.”
If the same cognitive processing holds, Gracie Gold and Julia Lipnitskaia might have been better holding something back just a little during the team competition. And Ashley Wagner may have unintentionally helped her cause. The world will soon find out.
NB: The umpire study is not easily found online, but for those interested here is the cite: David W. Rainey et al., The Effects of a Pitcher’s Reputation on Umpires’ Calls of Balls and Strikes, 12 Journal of Sport Behavior 139 (1989).
TrackBack URL for this entry:
Listed below are links to weblogs that reference Skating, Judging, and the Role of Direct Observation:
Wow. This is totally counter-intuitive. I would have expected the people who previously showed better control to get the benefit of the doubt on close calls, while the people who previously showed less control wouldn't.
Posted by: Howard Wasserman | Feb 10, 2014 11:23:03 AM
The baseball study is fascinating, but I'm not sure how well it will carry over to figure skating. Judging balls and strikes is a pretty limited inquiry; there's only so many ways a pitch can be one or the other, and umpire is judging repeated attempts of the same thing (hitting the strike zone), which sets a baseline percentage. But even a short program has 8 different required elements, spins, jumps, and footworks, and there's a lot of ways each can go wrong. Wagner might two-foot her landing on her double axel; or fall on her triple toe loop; or travel during her layback spin. Are the judges not going to call those things because she did them better earlier in the competition? That seems unlikely to me. They might not call these things because of other reasons, like pure nationalism, but I'm not sure one performance of each element sets any sort of baseline.
Posted by: Bruce Boyden | Feb 11, 2014 12:36:44 AM