BIB_08: Hughes, M. (1999). Rigor in usability testing. Technical Communication, 46(4), 488-494.

In this article, Hughes offers a succinct discussion on how to achieve rigor in quantitative and qualitative usability testing. Hughes starts with an overview of the basics of rigorous research, explaining key concepts in research methods such as validity (internal & external) and reliability as well as basics in statistics such as descriptive and inferential statistics. Recognizing the different strength and limitations of quantitative and qualitative methods, Hughes points out the benefits of using mixed methods -- using quantitative to "identify the possible problem area," and qualitative methods to "illustrate the reason or cause" (p. 491). Based on his discussion on rigor, Hughes then offers recommendations to ensure rigor using both quantitative and qualitative methods. With an emphasis on the rigor of qualitative research (because rigor is the most likely to be challenged using qualitative methods), he wraps up the article by offering more recommendations for usability testing such as using think-aloud protocol to ensure internal validity.

This article can serve as a guideline for method design and carrying out research in usability. Especially, it offers useful tools for qualitative researchers to conduct rigorous research. Hughes's recommendations for qualitative research validate qualitative research methods in usability testing, and allow technical communication researchers to conduct rigorous usability testing using methods we are familiar with. At the same time, the succinct explanation of the basic statistics can help those who are not familiar with quantitative methods understand these studies better and use quantitative methods appropriately. The discussion of mixed methods and the recommendation of triangulation also offer useful tools for researchers.