Usability Evaluations versus Usability Testing: When and Why?

Rosenbaum, S. Tec-Ed, Inc., Ann Arbor, MI

This paper appears in: Professional Communication, IEEE Transactions on Professional Communication
Publication Date: Dec 1989
Volume: 32, Issue: 4
On page(s): 210-216
ISSN: 0361-1434

To purchase the article: http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=44533

Abstract and Outline

This paper defines and distinguishes usability evaluations and usability testing, pointing out the role of expert evaluation in defining audience groups, constructing usage scenarios, and performing task breakdowns. Such evaluations are comparable to the work of an expert editor. The evaluation can justify usability testing, and can enable identification and prioritization of product aspects most likely to benefit from usability testing. Other methods besides testing that can supplement expert evaluation—surveys, focus groups, interviews, and user observations—are briefly discussed.

The article includes the following sections:

Who should define usability standards?
What background should usability specialists have?
Why not just test?

Drawbacks of very limited testing
Identifying appropriate usability testing

The expert evaluation: what is the process?

The audience/usage identification process
Adding general and specific usability data
Benefits and shortcomings of usability evaluations

How can you justify the usability test?

Controlling impact on product release
The importance of rigor

What alternatives to testing can supplement evaluations?
Conclusion

[Note: In today’s terminology, we would say “heuristic evaluation” rather than “usability evaluation,” because the latter is currently used as a general term to refer to many different methods, not just expert evaluation. The field of user experience has matured in many ways since this paper was published, but the paper still presents useful insights-as well as predating by several years other publications about heuristic evaluation.]