In this article, Macefield offers a guide to usability practitioners to determine the optimal participants group size in usability studies in commercial environment. The author maintains that usability studies literature should be applied with the consideration of the context of the study. With this basis, Macefield discusses the optimal participants group size in three types of usability studies, studies related to problem discovery, comparative studies, and punctuated studies. In discussion about studies related to problem discovery, Macefield considers (1) the problem discovery level and context criticality that is related to the severity of the problems, and (2) the complexity of the study. In addition, he discusses the case of prototype testing, which usually requires fewer participants than non-prototype testing. He summarizes that the participants group size must be determined considering these contexts, and 5-10 participants are "a sensible baseline range" (p. 39). In the his discussion on comparative studies, Macefield emphasizes the statistic significance of the differences between groups, which is determined by the effect size and the sample size. He concludes that although the determination of the group size can be arbitrary, it is again beneficial to consider the context of the study, and 8-25 participants per group is a sensible range, while 10-12 is a good range. Macefield describes studies that studying the participants in chunks as "punctuated studies." This type of studies, according to Macefield, promote the spirit of the iterative design process, and can lead to cost and time savings. Finally, as in almost all the article published in Usability Studies, Macefield provides a "practitioners take-away" list that summarizes his discussion for application.
This is a very good article to turn to when one designs her/his usability study. However, one has to bear in mind that the audience of this article is practitioners. That means that it is inevitably and rightly so pragmatic. When used in academic context, these guidelines need to be supported with literature and statistics. That said, Macefield's emphasis on contextualizing usability studies offers great insight to both academic researchers and practitioners. (And in spite of myself, I realized that Nielsen is a sort of the God Father of usability studies.)
Showing posts with label Nielsen. Show all posts
Showing posts with label Nielsen. Show all posts
Wednesday, October 13, 2010
BIB_08: Faulkner, L. (2003). Beyond the five-user assumption: Benefits of increased sample sizes in usability testing. Behavior Research Methods, Instrument, & Computers, 35(3), 379-383.
Faulkner's study challenges the "5-user assumption" (testing five users can identify 80% of all the problems) widely accepted and applied in usability studies. Faulkner conducted a usability study with 60 users, and analyzed data from sampled groups of different sizes. He found that testing five users can identify 55%-100% of problems, and increasing the sample size to ten significantly increased the minimum problems identified to 95%. Therefore, he argues that 5-user assumption should not be readily accepted, and testing 10 users can increase the confidence of the data and thus the validity of the testing.
This piece is a good example of an empirical study in usability. It raises the question of the sample size in usability testing and validity. The results can be used as a ground for future research design. It also shows how taking a recommendation or rule out of context (the 5-user assumption) and accepting it readily as applicable across the board can hurt the validity of research.
This piece is a good example of an empirical study in usability. It raises the question of the sample size in usability testing and validity. The results can be used as a ground for future research design. It also shows how taking a recommendation or rule out of context (the 5-user assumption) and accepting it readily as applicable across the board can hurt the validity of research.
Wednesday, September 22, 2010
RR_05: Cooper, A. (2004). The inmates are running the asylum. Indianapolis, IN: Sams.
Offering valuable insights in usable software-based products design, Cooper's book is foremost a rhetorical piece that advocates in the IT industry for a process of software-development that starts with interaction design preceding programming. On achieving this purpose, Cooper pushes for the methods for interaction design such as persona and goal-directed design. Although there are a few points I do not completely agree on, I found Cooper's work convincing and a useful resource for my project on cultural usability and Web 2.0.
First of all, Cooper's notion of interaction design is a useful concept worthy of our attention. Cooper distinguishes interaction design from interface design and prefers the former “because 'interface' suggests that you have code over here, people over there, and an interface in between that passes messages between them. It implies that only the interface is answerable to the users' needs” (p. 26). He goes on to argue about interface design that “[l]ike putting an Armani suit on Attila the Hun, interface design only tells how to dress up an existing behavior” (ibid). For an interaction designer, according to Cooper, the design of a product does not only involve interface design, but also behavioral design and conceptual design. He explains that “[b]ehavioral design tells how the elements of the software should act and communicate” (ibid). However, “[i]nteraction designers also work from the outside in, starting from the goals the user is trying to achieve, with an eye toward the broader goals of the business, the capabilities of the technology and the component tasks” (ibid). Thus it is important to include conceptual design, “which considers what is valuable for the users in the first place” (ibid), in the design process (ibid). Cooper argues that the order to the design process should start from conceptual design, then move on to behavioral design, and finally interface design (ibid).
Cooper's view on the interface is very narrow, which is limited to page layout or graphic design, something that “dresses up” the more crucial behaviors of application. Intentionally or unintentionally, Cooper trivializes interface design just as the way he trivializes the “aesthetic” element of design (p. 149).
There are a couple of prejudices and misconceptions involved in his views. First of all, the prejudice against “aesthetic” design that is subject to “subjective” judgment. Judgment of aesthetics is subjective to a certain extent, but that does not mean that such judgment is invalid or groundless. It is also dismissive and a bit pompous for Cooper to assume that design before the digital age only involved aesthetics but had nothing to do with “cognitive friction,” or “the resistance encountered by a human intellect when it engages with a complex system of rules that change as the problem changes” (p. 19). Think of architecture or industrial design, and Cooper's assumption is just a misconception that reinforces the misunderstanding of design as visual design that he himself opposes.
More importantly, to equal interface design as visual design with the only purpose for aesthetics is another misconception of Cooper's argument. The interface is often expressed visually, but it is more than just the “look” of the more heavy lifting code. Norman's (2002) notion of the “system image,” or the “physical structure” of the system (p. 16), can be used to understand the interface. According to Norman, the system image bridges the gap between what the designer designs (the design model) and what the user thinks how the system work (the user's model). Interface, as the physical thing where the interaction between the user and the machine happens is where the system image is materialized to the user. Therefore, interface design does not only involve the page layout, graphic design, or the looks of a system, but how the system is translated and presented to the user. If Cooper's concept of “cognitive friction” is useful at at all, we have to recognize that, contrary to what he claims, the interface is precisely where the cognitive friction occurs for the interface is the site where the human and the machine meet.
Despite these problematic claims, Cooper's insight is valuable in many ways. To be fair, his emphasis on the interaction over visual design (not interface design) seems to fit today's Web development better because of the increasing merging of Web sites and Web-based applications. More importantly, his work points out the problem with software-development processes many companies in the industry employ, and that is programming without interaction design. The product that often results from this process is what he calls a “dancing bear” which does not work well, but because the user does not have any other choices and has to get the job done, she/he has to tolerate the hard-to-use product. In this sense, Cooper is a champion advocate on behalf of the users. The process he advocates compared to the processes that do not work is illustrated as below (pp. 204-205):
From the illustrations, we can tell the Cooper would not agree with usability advocates such as Nielsen that user testing is the only way for good product design. Cooper points out some of the problems with user testing. First of all, he argues that “[t]he main reason why empirical user testing has been widely accepted in the high-tech business is that it fits easily into the existing sequence” (p. 204). That means that user testing actually reinforces the old process of software-development. One of the biggest challenges with user testing is that “[m]ost user testing depends on having a working program to test with, so necessarily it must wait until the program is up and running. This places user testing in the sequence conveniently in parallel to bug testing” (p. 204). This creates a problem, for “[n]o matter who does the designing, regardless of the method she might apply, the effect of design will be negligible if the coding is underway. A fundamental premise of bringing design to the software-development process is that it must occur before the programming begins” (p. 204).
Cooper argues that testing before programming, or what he calls the “pre-facto user testing” is “similar to the pure research that one would expect to find in a university setting” (p. 206). He uses an example of his colleague who tested the effectiveness of the status bar in the interface of a program. He argues that although the test result—users don't pay much attention to the status bar—is valuable, it does not “shed much insight on the underlying problems” (p. 206). Here again, the biggest challenge with pre-fecto user testing and testing or testing during the development of the product is that the product that gets tested is not yet built or completed, and thus any simulacrum (“either a quickly written prototype program or a 'puppet-show' made from paper cutouts or some equivalent, low-tech material” (p. 206)) used in the testing will result in inaccurate testing results.
Although he points out these problems with usability testing, Cooper does not completely dismiss its value. He agrees that “[t]houghtful user testing can uncover a designer's incorrect assumptions. Exposing your design work to users and then redesigning iteratively is always better than not doing so. Some new technologies […] are so untried that the insight provided by basic user testing can be of great value” (p. 207).
That said, usability testing to Cooper perhaps has more rhetorical value than engineering value:
Perhaps the most important and useful insight Cooper offers in his book is the idea of “goal-directed design” (p. 151). Unlike many others' work on usability (such as Nielsen or Barnum), Cooper distinguishes goals from tasks radically. He emphasizes that “[g]oals are not the same things as tasks. A goals is an end condition, whereas a task is an intermediate process needed to achieve the goal.” (p. 150). He then goes on and notes that “[t]here is an easy way to tell the difference between tasks and goals. Tasks change as technology changes, but goals have the pleasant property of remaining very stable” (p. 150). This means that the consideration of tasks is more closely linked to the product (and especially the features of the product), whereas the consideration of goals must focus on the user. Cooper argues that “[d]esigning from tasks instead of goals is one of the main causes of frustrating and ineffective interaction” and “asking, 'What are the user's goals?' lets us see through the confusion and create more appropriate and satisfactory designs” (p. 151). The goal-directed design requires the designers to analyze the user's goals to solve problems, which often can result in “very different—and much better—solutions” (p. 151).
The goal-directed design shifts the focus from technologies to the user. Especially, the personal goals are considered by Cooper as the most important goals the designer has to consider when designing a product. We can sense this approach in Norman's (2002) work, but Cooper is really pushing this idea forward in his work. The approach of goal-directed design, when applied to user research, will require the researcher to look at the original goals of the user, and then think of the system on the basis of these goals. When applied to research in cultural usability, the approach requires the researcher to shift from asking what color scheme on a Web site is more appealing to users from a certain culture, to understanding what are the important things a user wants to achieve (e.g., to fulfill her/his responsibilities for her/his parents in Confucian cultures), given her/his specific cultural context.
First of all, Cooper's notion of interaction design is a useful concept worthy of our attention. Cooper distinguishes interaction design from interface design and prefers the former “because 'interface' suggests that you have code over here, people over there, and an interface in between that passes messages between them. It implies that only the interface is answerable to the users' needs” (p. 26). He goes on to argue about interface design that “[l]ike putting an Armani suit on Attila the Hun, interface design only tells how to dress up an existing behavior” (ibid). For an interaction designer, according to Cooper, the design of a product does not only involve interface design, but also behavioral design and conceptual design. He explains that “[b]ehavioral design tells how the elements of the software should act and communicate” (ibid). However, “[i]nteraction designers also work from the outside in, starting from the goals the user is trying to achieve, with an eye toward the broader goals of the business, the capabilities of the technology and the component tasks” (ibid). Thus it is important to include conceptual design, “which considers what is valuable for the users in the first place” (ibid), in the design process (ibid). Cooper argues that the order to the design process should start from conceptual design, then move on to behavioral design, and finally interface design (ibid).
Cooper's view on the interface is very narrow, which is limited to page layout or graphic design, something that “dresses up” the more crucial behaviors of application. Intentionally or unintentionally, Cooper trivializes interface design just as the way he trivializes the “aesthetic” element of design (p. 149).
There are a couple of prejudices and misconceptions involved in his views. First of all, the prejudice against “aesthetic” design that is subject to “subjective” judgment. Judgment of aesthetics is subjective to a certain extent, but that does not mean that such judgment is invalid or groundless. It is also dismissive and a bit pompous for Cooper to assume that design before the digital age only involved aesthetics but had nothing to do with “cognitive friction,” or “the resistance encountered by a human intellect when it engages with a complex system of rules that change as the problem changes” (p. 19). Think of architecture or industrial design, and Cooper's assumption is just a misconception that reinforces the misunderstanding of design as visual design that he himself opposes.
More importantly, to equal interface design as visual design with the only purpose for aesthetics is another misconception of Cooper's argument. The interface is often expressed visually, but it is more than just the “look” of the more heavy lifting code. Norman's (2002) notion of the “system image,” or the “physical structure” of the system (p. 16), can be used to understand the interface. According to Norman, the system image bridges the gap between what the designer designs (the design model) and what the user thinks how the system work (the user's model). Interface, as the physical thing where the interaction between the user and the machine happens is where the system image is materialized to the user. Therefore, interface design does not only involve the page layout, graphic design, or the looks of a system, but how the system is translated and presented to the user. If Cooper's concept of “cognitive friction” is useful at at all, we have to recognize that, contrary to what he claims, the interface is precisely where the cognitive friction occurs for the interface is the site where the human and the machine meet.
Despite these problematic claims, Cooper's insight is valuable in many ways. To be fair, his emphasis on the interaction over visual design (not interface design) seems to fit today's Web development better because of the increasing merging of Web sites and Web-based applications. More importantly, his work points out the problem with software-development processes many companies in the industry employ, and that is programming without interaction design. The product that often results from this process is what he calls a “dancing bear” which does not work well, but because the user does not have any other choices and has to get the job done, she/he has to tolerate the hard-to-use product. In this sense, Cooper is a champion advocate on behalf of the users. The process he advocates compared to the processes that do not work is illustrated as below (pp. 204-205):
From the illustrations, we can tell the Cooper would not agree with usability advocates such as Nielsen that user testing is the only way for good product design. Cooper points out some of the problems with user testing. First of all, he argues that “[t]he main reason why empirical user testing has been widely accepted in the high-tech business is that it fits easily into the existing sequence” (p. 204). That means that user testing actually reinforces the old process of software-development. One of the biggest challenges with user testing is that “[m]ost user testing depends on having a working program to test with, so necessarily it must wait until the program is up and running. This places user testing in the sequence conveniently in parallel to bug testing” (p. 204). This creates a problem, for “[n]o matter who does the designing, regardless of the method she might apply, the effect of design will be negligible if the coding is underway. A fundamental premise of bringing design to the software-development process is that it must occur before the programming begins” (p. 204).
Cooper argues that testing before programming, or what he calls the “pre-facto user testing” is “similar to the pure research that one would expect to find in a university setting” (p. 206). He uses an example of his colleague who tested the effectiveness of the status bar in the interface of a program. He argues that although the test result—users don't pay much attention to the status bar—is valuable, it does not “shed much insight on the underlying problems” (p. 206). Here again, the biggest challenge with pre-fecto user testing and testing or testing during the development of the product is that the product that gets tested is not yet built or completed, and thus any simulacrum (“either a quickly written prototype program or a 'puppet-show' made from paper cutouts or some equivalent, low-tech material” (p. 206)) used in the testing will result in inaccurate testing results.
Although he points out these problems with usability testing, Cooper does not completely dismiss its value. He agrees that “[t]houghtful user testing can uncover a designer's incorrect assumptions. Exposing your design work to users and then redesigning iteratively is always better than not doing so. Some new technologies […] are so untried that the insight provided by basic user testing can be of great value” (p. 207).
That said, usability testing to Cooper perhaps has more rhetorical value than engineering value:
Arguably, the most valuable contribution of usability testing is made when programmers are forced to sit behind the one-way mirrors to view typical users struggling with their programs. The programmers shocked and incredulous, shouting sentiments like, “You are testing mental retards!” Usability testing is a useful whack on the side of the head of recalcitrant software engineers, showing them that there is indeed a problem. It can serve the same purpose for management, too. (p. 207)Considering that Cooper's mean concern in the book is how to persuade companies (and the personnel in the companies from the management to programmers) to embrace the development process that incorporate interaction design before programming, his view on user testing should not be thought as dismissive. However, I'm not sure if he does justice to the user testing method in usability studies.
Perhaps the most important and useful insight Cooper offers in his book is the idea of “goal-directed design” (p. 151). Unlike many others' work on usability (such as Nielsen or Barnum), Cooper distinguishes goals from tasks radically. He emphasizes that “[g]oals are not the same things as tasks. A goals is an end condition, whereas a task is an intermediate process needed to achieve the goal.” (p. 150). He then goes on and notes that “[t]here is an easy way to tell the difference between tasks and goals. Tasks change as technology changes, but goals have the pleasant property of remaining very stable” (p. 150). This means that the consideration of tasks is more closely linked to the product (and especially the features of the product), whereas the consideration of goals must focus on the user. Cooper argues that “[d]esigning from tasks instead of goals is one of the main causes of frustrating and ineffective interaction” and “asking, 'What are the user's goals?' lets us see through the confusion and create more appropriate and satisfactory designs” (p. 151). The goal-directed design requires the designers to analyze the user's goals to solve problems, which often can result in “very different—and much better—solutions” (p. 151).
The goal-directed design shifts the focus from technologies to the user. Especially, the personal goals are considered by Cooper as the most important goals the designer has to consider when designing a product. We can sense this approach in Norman's (2002) work, but Cooper is really pushing this idea forward in his work. The approach of goal-directed design, when applied to user research, will require the researcher to look at the original goals of the user, and then think of the system on the basis of these goals. When applied to research in cultural usability, the approach requires the researcher to shift from asking what color scheme on a Web site is more appealing to users from a certain culture, to understanding what are the important things a user wants to achieve (e.g., to fulfill her/his responsibilities for her/his parents in Confucian cultures), given her/his specific cultural context.
Tuesday, August 31, 2010
RR_02: Nielsen, J., & Loranger, H. (2006). Prioritizing Web usability. Berkeley, Calif.: New Riders.
Nielsen and Loranger's book is perhaps one of the most well researched and comprehensive Web usability on the market. The information delivered in the book—both the discovery of users' behaviors and the guidelines for Web usability design—is very useful. For instance, the severity scores of the usability problems draw attention to the careful design for better search, findability (IA, category names, navigation, links), page design (readability, layout, graphics, amateur, scrolling), and information (content, product info, corporate info, prices) (p. 131). Despite that the book is both a market success and a useful reference for designers, there are a few issues with the book that need to be addressed.
Continuing the point of severity, I found the concept of “severity” not a very useful attribute because of its complexity. According to Nielsen, the severity of a usability problem is determined by its frequency, impact, and persistence (p. 125). The calculation of the severity score based on the scores of the three attributes (rated on a 10-point scale) is to “multiply the frequency rating by the impact rating, then multiply that number by the square root of the persistence rating and divide that by the square root of 10” (p. 123). Granted that factoring in three attributes and comes up with a single score of severity is a very attractive idea and is useful to a certain extent (such as to give an overall idea of what are the most severe areas of usability problems as the authors have done), I do not think this score can be of practical use when guiding the design, because the three attributes, frequency, impact, and persistence really describe different aspects of a problem and need to be considered holistically in a real situation when a designer or a usability specialist are trying do decide which problems are to be fixed under the specific financial, technological, and personnel conditions. In other words, in a real design situation, using a single value to determine what usability problems should be prioritized is problematic. In fact, the authors surely understand the messiness of the prioritizing usability problems in a practice. Nielsen and Loranger have pointed out that severity should not be the only factor when making the decision of what problems to fix: “when enough little irritations add up to a bad user experience, people are likely to leave and not return” (p. 132). However, they failed to offer a better method of application of their data. In fact, providing a discussion of how to consider and balance the treatment of problems based on the three attribute rather than simply meshing them together into one single attribute will be more helpful in a practical situation.
Another issue with the book is related to cultural usability and Web localization. Although the authors claim that their research is based on testing with users from all over the world, the guidelines still fail to address possible cultural differences in Web usability design. In fact, the guidelines are more suitable for cultures of, using Hofstede's and Hall's cultural categories, high individualism, low power distance, high masculinity, and low-context, characteristics of the U.S. (see Singh & Pereira, 2005).
For instance, when analyzing a Web site with “high-severity usability problem,” the authors write, “The problem on this bank's "About Us" page is that it does not tell enough to establish trust and credibility. Yes, the bank says that it is a "home of traditional banking," but it doesn't back that up with facts such as when the bank was founded, how many branches it has, how solid it is, or any other specific information that would make you feel comfortable handing your money over to it” (p. 127). What the authors have pointed out is surely a usability problem. However, some would argue that this problem is considered more severe in the U.S., a masculine, highly individualist culture, and low-contextual, where statistics and facts are highly valued, than in more feminine, collective culture, and high-contextual, where community and family are valued more than hard numbers. The language on the Web site such as “take advantage of nearly 150 years of banking experience,” and “We're proud of our history of helping thousands of families and individuals to save money, buy homes, plan for a wedding or college education, prepare for retirement, and to make banking—and everyday life—easier and more convenient” is very appealing to an audience that values community and family. The images of seniors and a middle-aged couple are also appealing to a high-contextual culture. On the other hand, the problem with the lack of specific statistics might not be a huge problem in high-contextual cultures. In fact, according to Singh & Pereira, politeness and indirectness are highly valued in high-contextual cultures, and “the use of indirect words like 'perhaps,' 'probably,' and 'somewhat'” can emphasize the politeness on Web sites in these cultures (p. 145). Throughout the book, issues of cultural specificity have never been brought up, which is a disappointment.
This problem can be seen as part of an larger issue with this book, namely the logocentric tendency that noted by scholars concerned with the importance of context in usability. As Sun (2004) has pointed out in his dissertation Expending the Scope of Localization: “Nielsen’s view of usability attends to issues within a narrow scope: Contextual factors such as social and cultural aspects are detached from the use of product functionality. This view favors the system and defines usability as attributes measured by quantitative methods” (p. 37). This tendency can be seen throughout the book. For instance, the extensive use of statistics, and the emphasis on directness, efficiency, and simplicity. These characteristics are not necessarily bad things—quite on the contrary, they add much strength to the book—but not recognizing the complexity of contexts is a weakness in the book.
Along the same line, the book emphasizes on e-Commerce and business Web sites more than anything else. Although the authors also mention Web sites such as hospital and government Web sites, their emphasis on business makes usability seem limited to retaining users and “converting” them from users into consumers. At the Computer and Writing 2010 conference, researchers Michele Simmons (Miami University) and Meredith Zoetewey (University of South Florida) presented their research on usability of civic engagement. In their presentation, they pointed out the limitation of Nielsen's usability that is primarily for businesses, and argued that for civic Web sites, simple and fast information design could not meet the requirements of citizens who sought more complex, complete, and comprehensive information. They proposed that Nielsen's guidelines of usability should be revised for civic Web site. I completely agree with them, and want to add that this is also true with educational Web sites, where certain challenge for users/students should be part of the design.
The last point I want to make is that the book, although updated from its 1994 version, lacks a discussion on the usability in Web 2.0 environment that has been becoming the Web itself. The only place where the authors mentioned Web 2.0 is in the preface when they talk about the fact that their usability guidelines do not apply to teens' MySpace profile page design. I agree with them on this point, but there is more to Web sites and applications such as MySpace, RSS, Facebook, Wiki, Twitter, YouTube and many others which are called by some Web 2.0 Web sites/applications. I will discuss Web 2.0 with more depth in later posts. The point is, published in 2006, one would assume that this book would mention at least some aspects of the Web we have today, but it seems the Nielsen and Loranger still stay in the e-Commerce era.
References
Singh, N., & Pereira, A. (2005). The culturally customized Web site. Burlington, MA: Elsevier.
Sun, H. (2004). Expanding the scope of localization: A cultural usability perspective on mobile text messaging use in American and Chinese contexts. Rensselaer Polytechnic Institute, Troy, New York.
Continuing the point of severity, I found the concept of “severity” not a very useful attribute because of its complexity. According to Nielsen, the severity of a usability problem is determined by its frequency, impact, and persistence (p. 125). The calculation of the severity score based on the scores of the three attributes (rated on a 10-point scale) is to “multiply the frequency rating by the impact rating, then multiply that number by the square root of the persistence rating and divide that by the square root of 10” (p. 123). Granted that factoring in three attributes and comes up with a single score of severity is a very attractive idea and is useful to a certain extent (such as to give an overall idea of what are the most severe areas of usability problems as the authors have done), I do not think this score can be of practical use when guiding the design, because the three attributes, frequency, impact, and persistence really describe different aspects of a problem and need to be considered holistically in a real situation when a designer or a usability specialist are trying do decide which problems are to be fixed under the specific financial, technological, and personnel conditions. In other words, in a real design situation, using a single value to determine what usability problems should be prioritized is problematic. In fact, the authors surely understand the messiness of the prioritizing usability problems in a practice. Nielsen and Loranger have pointed out that severity should not be the only factor when making the decision of what problems to fix: “when enough little irritations add up to a bad user experience, people are likely to leave and not return” (p. 132). However, they failed to offer a better method of application of their data. In fact, providing a discussion of how to consider and balance the treatment of problems based on the three attribute rather than simply meshing them together into one single attribute will be more helpful in a practical situation.
Another issue with the book is related to cultural usability and Web localization. Although the authors claim that their research is based on testing with users from all over the world, the guidelines still fail to address possible cultural differences in Web usability design. In fact, the guidelines are more suitable for cultures of, using Hofstede's and Hall's cultural categories, high individualism, low power distance, high masculinity, and low-context, characteristics of the U.S. (see Singh & Pereira, 2005).
For instance, when analyzing a Web site with “high-severity usability problem,” the authors write, “The problem on this bank's "About Us" page is that it does not tell enough to establish trust and credibility. Yes, the bank says that it is a "home of traditional banking," but it doesn't back that up with facts such as when the bank was founded, how many branches it has, how solid it is, or any other specific information that would make you feel comfortable handing your money over to it” (p. 127). What the authors have pointed out is surely a usability problem. However, some would argue that this problem is considered more severe in the U.S., a masculine, highly individualist culture, and low-contextual, where statistics and facts are highly valued, than in more feminine, collective culture, and high-contextual, where community and family are valued more than hard numbers. The language on the Web site such as “take advantage of nearly 150 years of banking experience,” and “We're proud of our history of helping thousands of families and individuals to save money, buy homes, plan for a wedding or college education, prepare for retirement, and to make banking—and everyday life—easier and more convenient” is very appealing to an audience that values community and family. The images of seniors and a middle-aged couple are also appealing to a high-contextual culture. On the other hand, the problem with the lack of specific statistics might not be a huge problem in high-contextual cultures. In fact, according to Singh & Pereira, politeness and indirectness are highly valued in high-contextual cultures, and “the use of indirect words like 'perhaps,' 'probably,' and 'somewhat'” can emphasize the politeness on Web sites in these cultures (p. 145). Throughout the book, issues of cultural specificity have never been brought up, which is a disappointment.
This problem can be seen as part of an larger issue with this book, namely the logocentric tendency that noted by scholars concerned with the importance of context in usability. As Sun (2004) has pointed out in his dissertation Expending the Scope of Localization: “Nielsen’s view of usability attends to issues within a narrow scope: Contextual factors such as social and cultural aspects are detached from the use of product functionality. This view favors the system and defines usability as attributes measured by quantitative methods” (p. 37). This tendency can be seen throughout the book. For instance, the extensive use of statistics, and the emphasis on directness, efficiency, and simplicity. These characteristics are not necessarily bad things—quite on the contrary, they add much strength to the book—but not recognizing the complexity of contexts is a weakness in the book.
Along the same line, the book emphasizes on e-Commerce and business Web sites more than anything else. Although the authors also mention Web sites such as hospital and government Web sites, their emphasis on business makes usability seem limited to retaining users and “converting” them from users into consumers. At the Computer and Writing 2010 conference, researchers Michele Simmons (Miami University) and Meredith Zoetewey (University of South Florida) presented their research on usability of civic engagement. In their presentation, they pointed out the limitation of Nielsen's usability that is primarily for businesses, and argued that for civic Web sites, simple and fast information design could not meet the requirements of citizens who sought more complex, complete, and comprehensive information. They proposed that Nielsen's guidelines of usability should be revised for civic Web site. I completely agree with them, and want to add that this is also true with educational Web sites, where certain challenge for users/students should be part of the design.
The last point I want to make is that the book, although updated from its 1994 version, lacks a discussion on the usability in Web 2.0 environment that has been becoming the Web itself. The only place where the authors mentioned Web 2.0 is in the preface when they talk about the fact that their usability guidelines do not apply to teens' MySpace profile page design. I agree with them on this point, but there is more to Web sites and applications such as MySpace, RSS, Facebook, Wiki, Twitter, YouTube and many others which are called by some Web 2.0 Web sites/applications. I will discuss Web 2.0 with more depth in later posts. The point is, published in 2006, one would assume that this book would mention at least some aspects of the Web we have today, but it seems the Nielsen and Loranger still stay in the e-Commerce era.
References
Singh, N., & Pereira, A. (2005). The culturally customized Web site. Burlington, MA: Elsevier.
Sun, H. (2004). Expanding the scope of localization: A cultural usability perspective on mobile text messaging use in American and Chinese contexts. Rensselaer Polytechnic Institute, Troy, New York.
Monday, August 30, 2010
BIB_02: Nielsen, J., & Loranger, H. (2006). Prioritizing Web usability. Berkeley, Calif.: New Riders.
Nielsen and Loranger's book is one of the most cited Web usability books. The guidelines in the book are based on the data of the usability testing (716 Web sites with 2,163 users) conducted by Nielsen's usability consulting firm Nielsen Norman Group and additional testing (25 Web sites and 69 users) specifically for this book. The authors revised their older guideline published in 1994, and discuss how to prioritize usability problems. The following seven chapters are dedicated to guidelines for specific issues including search, navigation and IA, typography, writing for the Web, product information, visual design, and the use of multimedia. In the final chapter, the authors stress the importance of understanding the target users and prioritizing function before form, and encourage designers to design usable and elegant Web sites.
This book covers many basic Web usability issues with solid data and confident argument. In fact, the shear extensiveness of their data and experience makes their guidelines credible and convincing. In addition, the book includes a large number of examples with attractive and faithful visual presentation (color images of Web sites), and the authors offers' analysis and evaluation, which I found not only helpful for the readers to understand the material, but also very appealing. Despite that, I found two major problems with the book. First, although the authors claim that the guidelines in the book apply to the Web sites of large, medium and small corporations, eCommerce, governments, and other non-profit organizations, the book has a strong inclination towards eCommerce and corporations. Little attention is given to Web sits of purposes other than those to sell. Second, the authors have not discussed at all Web 2.0 applications/Web sites, which are becoming... the Web. The book was published in 2006, when Web 2.0 was starting to get attention in the industry. The absence of mentioning of Web 2.0 is indeed an appointment.
This book covers many basic Web usability issues with solid data and confident argument. In fact, the shear extensiveness of their data and experience makes their guidelines credible and convincing. In addition, the book includes a large number of examples with attractive and faithful visual presentation (color images of Web sites), and the authors offers' analysis and evaluation, which I found not only helpful for the readers to understand the material, but also very appealing. Despite that, I found two major problems with the book. First, although the authors claim that the guidelines in the book apply to the Web sites of large, medium and small corporations, eCommerce, governments, and other non-profit organizations, the book has a strong inclination towards eCommerce and corporations. Little attention is given to Web sits of purposes other than those to sell. Second, the authors have not discussed at all Web 2.0 applications/Web sites, which are becoming... the Web. The book was published in 2006, when Web 2.0 was starting to get attention in the industry. The absence of mentioning of Web 2.0 is indeed an appointment.
Monday, August 23, 2010
RR_01: Barnum, C. M. (2002). Usability testing and research. New York: Longman.
Barnum's text book, although maybe a bit dated, is a great start for usability students. It is both a theory book that maps out the theory and key issues of usability research and user-centered design (UCD), and a how-to book that meticulously explains how to conduct usability testing and create documents for the testing. I'll discuss a few issues that interest me or relate to my dissertation project below.
First of all, Barnum summarizes nicely the definition of usability and UCD. She draws on the definitions of usability by key sources/writers/researchers such as the Internationalization Organization for Standardization, Nielsen, and Dumas and Redish, all widely cited in usability studies literature, and points out that the focus of usability studies is the user. To make her point, she defines usability negatively, as not (p. 6):
Pointing out that “[t]hese terms reflect issues related to the product itself, not to the interaction of a person with the product” (p. 6), she draws attention to the distinction between product quality and usability. She further explains that usability has two requirements, “usefulness” and “satisfaction,” and contends that usability is determined by the user's perception of the quality of the product (p. 6-7). Two points are worth discussing here. First, her notion of usefulness is a key concept when we consider the split between task-oriented and goal-oriented usability studies. While task-oriented usability focuses on designing usable products, the goal-oriented usability focuses on designing useful products.
Although the terms “task” and “goal” are often used interchangeably by some authors, and the distinction between these two is not always clear in a lot of literature, this split can been seen among usability researchers and practitioners such as Nielsen, Cooper, and Krug. Barnum seems to agree with Cooper's view that we should design for the user's goals instead of tasks. When later on discussing task analysis, Barnum lists the goals of task analysis as (p. 88):
This list shows the distinction between goals and tasks. The goals are what users want to achieve in their life, such as succeeding a course. The tasks are more specific than goals. They are the means to goal, i.e., what the user needs to do to achieve her/his goals. In the example of the user's goal being succeeding a course, one of the tasks will be buying textbooks online. The users have multilevel goals, so the task of buying textbooks online are also means to achieve other goals such as saving money, and saving gas going to a bookstore and time browsing there. Understanding these goals, articulated most eloquently in Cooper's The Inmates Are Running the Asylum, is essential in designing useful and usable products for the users. The goal-oriented approach is useful in cultural usability research for it can yield rich data about users due to its focus on their goals that are necessarily formed in their specific context of use.
Second, Barnum's argument of perception can be controversial. Although this book focuses on usability testing, Barnum's approach to usability research is inclusive in terms of methods. She talks about different methods in UCD in which the researcher or designer interact with users in different ways. These methods include focus group, prototyping, interviews, survey, etc. These methods are designed to get feedback from users. In other words, they are designed to obtain data about the user's perception of the product's usability. However, some hardliner usability testing proponents such as Nielsen and Krug will argue that using these “soft” methods, usability issues or usability breakdowns cannot be revealed accurately or adequately. They argue that empirical usability testing is the only way to know if a product works for the user. The tension between these two views is not surprising if we take the researchers' backgrounds into account. Barnum is a technical communication scholar who has a background in humanities (rhetoric). Nielsen and Krug both have engineering background on their resume. I will argue that methods that originated in humanities and those in sciences can help us obtain different data to understand the user and her/his interaction with technologies from different angles. Especially in today's complex technological environment where technologies are so profoundly entrenched on everyone's life, pure empirical methods may not be adequate to collect rich and nuance data about users.
The last thing I want to discuss about this book is its chapter on Web usability (chapter 9). To be honest, I'm a bit disappointed with this chapter. Not only the material is dated (which however is understandable because of its subject matter), but it seems that the whole chapter is pretty much a rip off of Nielsen's work. Also, Barnum does not seem to have carefully thought through Web usability issues. For instance, when she talks about understanding the real users' needs, she summarizes the users' reasons to use the Web into three groups, “information (content), sales (commerce), and interaction (communication with other people)” (p. 365). The grouping is not only crude but also leaves out many other reasons (goals) for users to use the Web, such as entertainment, (personal, political...) expression, advocacy and mobilization, etc. These groups are so broad that I suspect that they are actually of any productive value. She may argue that reasons such as expression and advocacy can be grouped under interaction. If that's the case, I will argue that everything can be grouped under information, because ultimately the Web is information. Barnum here is a bit self-contradictory—as an advocate of designing for users' goals, and the individual needs of users, her crude grouping does seem a bit careless.
First of all, Barnum summarizes nicely the definition of usability and UCD. She draws on the definitions of usability by key sources/writers/researchers such as the Internationalization Organization for Standardization, Nielsen, and Dumas and Redish, all widely cited in usability studies literature, and points out that the focus of usability studies is the user. To make her point, she defines usability negatively, as not (p. 6):
- Quality assurance
- Zero defects
- Utility of design features
- Intrinsic in products
Pointing out that “[t]hese terms reflect issues related to the product itself, not to the interaction of a person with the product” (p. 6), she draws attention to the distinction between product quality and usability. She further explains that usability has two requirements, “usefulness” and “satisfaction,” and contends that usability is determined by the user's perception of the quality of the product (p. 6-7). Two points are worth discussing here. First, her notion of usefulness is a key concept when we consider the split between task-oriented and goal-oriented usability studies. While task-oriented usability focuses on designing usable products, the goal-oriented usability focuses on designing useful products.
Although the terms “task” and “goal” are often used interchangeably by some authors, and the distinction between these two is not always clear in a lot of literature, this split can been seen among usability researchers and practitioners such as Nielsen, Cooper, and Krug. Barnum seems to agree with Cooper's view that we should design for the user's goals instead of tasks. When later on discussing task analysis, Barnum lists the goals of task analysis as (p. 88):
- What users' goals are, not just the tasks they perform
- What processes they us to achieve their goals
- What characteristics shape the way they perform tasks and achieve goals (different groups of users may have different characteristics and goals)
- What previous experience shapes users' approaches to tasks
- What is most important to users or what is most helpful to them in performing tasks
- What impact the environment has on their ability to perform tasks
This list shows the distinction between goals and tasks. The goals are what users want to achieve in their life, such as succeeding a course. The tasks are more specific than goals. They are the means to goal, i.e., what the user needs to do to achieve her/his goals. In the example of the user's goal being succeeding a course, one of the tasks will be buying textbooks online. The users have multilevel goals, so the task of buying textbooks online are also means to achieve other goals such as saving money, and saving gas going to a bookstore and time browsing there. Understanding these goals, articulated most eloquently in Cooper's The Inmates Are Running the Asylum, is essential in designing useful and usable products for the users. The goal-oriented approach is useful in cultural usability research for it can yield rich data about users due to its focus on their goals that are necessarily formed in their specific context of use.
Second, Barnum's argument of perception can be controversial. Although this book focuses on usability testing, Barnum's approach to usability research is inclusive in terms of methods. She talks about different methods in UCD in which the researcher or designer interact with users in different ways. These methods include focus group, prototyping, interviews, survey, etc. These methods are designed to get feedback from users. In other words, they are designed to obtain data about the user's perception of the product's usability. However, some hardliner usability testing proponents such as Nielsen and Krug will argue that using these “soft” methods, usability issues or usability breakdowns cannot be revealed accurately or adequately. They argue that empirical usability testing is the only way to know if a product works for the user. The tension between these two views is not surprising if we take the researchers' backgrounds into account. Barnum is a technical communication scholar who has a background in humanities (rhetoric). Nielsen and Krug both have engineering background on their resume. I will argue that methods that originated in humanities and those in sciences can help us obtain different data to understand the user and her/his interaction with technologies from different angles. Especially in today's complex technological environment where technologies are so profoundly entrenched on everyone's life, pure empirical methods may not be adequate to collect rich and nuance data about users.
The last thing I want to discuss about this book is its chapter on Web usability (chapter 9). To be honest, I'm a bit disappointed with this chapter. Not only the material is dated (which however is understandable because of its subject matter), but it seems that the whole chapter is pretty much a rip off of Nielsen's work. Also, Barnum does not seem to have carefully thought through Web usability issues. For instance, when she talks about understanding the real users' needs, she summarizes the users' reasons to use the Web into three groups, “information (content), sales (commerce), and interaction (communication with other people)” (p. 365). The grouping is not only crude but also leaves out many other reasons (goals) for users to use the Web, such as entertainment, (personal, political...) expression, advocacy and mobilization, etc. These groups are so broad that I suspect that they are actually of any productive value. She may argue that reasons such as expression and advocacy can be grouped under interaction. If that's the case, I will argue that everything can be grouped under information, because ultimately the Web is information. Barnum here is a bit self-contradictory—as an advocate of designing for users' goals, and the individual needs of users, her crude grouping does seem a bit careless.
Subscribe to:
Posts (Atom)