BIB_08: Faulkner, L. (2003). Beyond the five-user assumption: Benefits of increased sample sizes in usability testing. Behavior Research Methods, Instrument, & Computers, 35(3), 379-383.

Faulkner's study challenges the "5-user assumption" (testing five users can identify 80% of all the problems) widely accepted and applied in usability studies. Faulkner conducted a usability study with 60 users, and analyzed data from sampled groups of different sizes. He found that testing five users can identify 55%-100% of problems, and increasing the sample size to ten significantly increased the minimum problems identified to 95%. Therefore, he argues that 5-user assumption should not be readily accepted, and testing 10 users can increase the confidence of the data and thus the validity of the testing.

This piece is a good example of an empirical study in usability. It raises the question of the sample size in usability testing and validity. The results can be used as a ground for future research design. It also shows how taking a recommendation or rule out of context (the 5-user assumption) and accepting it readily as applicable across the board can hurt the validity of research.