In one of the best-known scenes of Milos Forman’s 1984 Academy Award winning film, Amadeus, the young Mozart upstages his rival, the composer Antonio Salieri with a performance of the “Abduction from Seraglio.” After the performance, Mozart is filled with pride and positively twitching with excitement as the Austrian Emperor, Joseph II, comes forward to share his congratulations. Mozart’s wide smile evaporates, however, when the emperor punctuates his compliments with a bit of regal advice: “There are too many notes.” Mozart, stunned, replies, “I don’t understand. There are just as many notes, Majesty, as are required. Neither more nor less.” To which the emperor responds, somewhat patronizingly, “There are simply too many notes. Cut a few and it will be perfect.”
This moment of cinematic history offers an important insight into the relationships between quality and quantity. In some cases, the effort to improve the quality of an activity or program by tinkering with one of its quantitative aspects represents a form of category mistake, like telling a painter that she should use less red. In other cases, however, the Emperor Josephs of the world may have an important point to make. That is, one of the best opportunities to enhance quality is by tinkering with quantity. For example, there is pretty clear evidence that the quality of relationships such as marriage depends in part on the quality of the conversation between spouses. In general, spouses in strong marriages speak more words to one another on a daily basis than those in failing and doomed relationships.
We believe that a similar lesson applies to one of the education world’s most prized assessment techniques, the Likert scale. Developed by psychologist Rensis Likert (1903–1981) as part of his doctoral thesis at Columbia University, the scale was intended to collect deeper information about attitudes than the competing methods of the time. Even people without a strong background in educational assessment are familiar with Likert’s 5-point scale, with response options ranging from “strongly disagree” to “disagree” to “neutral” to “agree” to “strongly agree.” Interestingly, Likert himself, though the inventor of his eponymous scale, frequently argued for the use of open-ended questions in his work. Instead of requiring respondents to choose from a scale of preestablished alternatives, respondents could respond using their own words. However, Likert’s principal legacy in this regard is the closed-ended Likert scale.
Why did Likert express dissatisfaction with his own intellectual offspring? We hypothesize that the problem was a fundamental quantitative one and, in particular, a quantitative insufficiency that soon takes on the appearance of a qualitative defect. Two of the best-known drawbacks of the Likert scale arise from its original 5-point design. First, it is not always clear that the difference between “strongly disagree” and “disagree” is quantitatively the same as the difference between “disagree” and “neutral.” Second, critiques of the scale point out that respondents may frequently find themselves in the situation of needing an option somewhere between those proffered by the traditional 5 points. For example, respondents may feel sufficiently ambivalent toward a particular statement that they need some option between “disagree” and “neutral,” such as “mildly disagree.” While Mozart’s work had too many notes, Likert’s offered too few options.
By adding more options to the traditional 5-point Likert scale, we can address many of its long-standing deficiencies. For example, when a learner assesses a lecture, an educator assesses a learner, or an outside expert assesses an educational program, we should provide them with more than five options on every item. Consider the following item from an educational assessment. “The educator made effective use of visual aids.” The traditional Likert item would offer five options, from “strongly disagree” to “strongly agree.” In place of this rather simplistic, bare-bones approach to item writing, we propose a far more sophisticated, nuanced, and robust replacement, with the following response options: “very strongly disagree,” “strongly disagree,” “mildly strongly disagree,” “disagree,” “strongly mildly disagree,” “mildly disagree,” “neutral,” “mildly agree,” “strongly mildly agree,” “agree,” “mildly strongly agree,” “strongly agree,” and “very strongly agree.”
This new and improved 13-point version overcomes many of the traditional weaknesses of the Likert scale. First, it offers so many quantitative options for interpolation that it effectively turns a quantitative assessment into a quasi-qualitative assessment. Second, we provide respondents with an opportunity more accurately and precisely to capture their attitudes toward each statement. No longer must they merely disagree or strongly disagree. Now they can very strongly disagree or only disagree mildly. By decreasing the gaps between response items, we can produce much smoother curves of response distributions. To use an illustration familiar to radiologists, this new methodology will enable our results summaries to appear less pixilated than they did in Likert’s day.
Get Radiology Tree app to read full this article<