Concurrent versus Retrospective Verbal Protocol for Comparing Window Usability

Abstract

A traditional concurrent verbal protocol method was compared to a heavily cued retrospective verbal protocol in which users were presented with a video tape of their performance to help them recall their thoughts after task completion. The two methods of protocol were employed in a comparison of two different size monitors. Subjects were required to complete 12 tasks which varied in the number of windows required simultaneously on the monitor. The subjects' performance, as measured by steps to completion, task completion time, and errors committed, was compared across monitors and protocol methods. Subjective data were also collected in the form of task difficulty ratings, as well as a global measure of user satisfaction. Verbal data were compared to assess any information differences due to the methods of collection or the monitor sizes. No performance or subjective differences were found between the two protocol methods. The kinds of information gathered were quite different for the two methods, with concurrent protocol subjects giving procedural information and retrospective protocol subjects giving explanations and design statements. Performance data, as well as subjective data, indicated that on tasks that require that one or two windows be present simultaneously, there were no differences between the two monitor sizes. As the number of simultaneous windows increased, however, the large monitor's advantages became apparent. Tasks which require that four windows be present simultaneously were judged to be easier and required fewer steps on the large monitor than on the small monitor.