« Scholarship Matters to the Court…in Federal Civil Rulemaking…Maybe, Kind of, Sort of | Main | Inaugural Post »

Tuesday, May 05, 2015

Do Students Perform Better When Your Test Is a "Little Bit Harder" to Read?

It's an unusual exam-time question.  But according to this newly-released study, the answer is "no."

The question was prompted by a fascinating, well-publicized experiment in 2007 that found that people score higher on tests when the questions are very hard to read. When students took a particular test with a normal font, 90% made at least one mistake on the test. But that proportion dropped to 35% when the font was barely legible.  The experiment received a lot of attention.  It has been cited in over 130 articles, and Nobel Laureate Daniel Kahneman highlighted the findings in his 2011 book, Thinking, Fast and Slow. Malcolm Gladwell similarly emphasized the benefits of tests that are "just a little bit harder to read."

The idea that difficult tasks can "kick our brains into higher gear" is consistent with many ideas in cognitive psychology.  Cognitive psychologists have identified two kinds of decisionmaking processes: intuitive and deliberative.  Intuitive decisionmaking processes, called System I processes, are intuitive, automatic, and quick, encompassing the types of instantaneous judgments that permit a person to immediately size up a situation. Deliberative processes, or System II processes, describe reflective, logical, and self-conscious thinking. See Adam S. Zimmerman, Funding Irrationality, 59 Duke L.J. 1105 (2010).  Kahneman and others have long suggested that deliberative processing can “override” System I processes under certain circumstances--which is why people are less susceptible to cognitive errors or biases when there are opportunities to learn from experience or when they have access to third-party expertise, like lawyers, doctors, or other specialists.  For that reason, regulatory efforts to "de-bias through law" often rely on rules that encourage people to reflect or deliberate more about their choices to improve welfare. 

After surveying results from over 7,000 people, however, Terry Burnham, Shane Frederick, Andrew Meyers and eight other co-authors, however, appear to have refuted this particular study.  The paper appears in the April 2015 Journal of Experimental Psychology.    

Posted by Adam Zimmerman on May 5, 2015 at 02:35 PM in Science | Permalink


Nicely said. And changed.

Posted by: Adam Zimmerman | May 6, 2015 9:30:55 AM

"When 90% of the students saw a question in a normal font"

Not sure if just poorly phrased ...or Zimmerman is trying to kick our brains into higher gear.

Posted by: Derek Tokaz | May 6, 2015 8:53:35 AM

The comments to this entry are closed.