Making Usability a Priority: Advocating for the Value of User Research

By Cory Lebson

As a consultant specializing in user research, studying how users interact with Web and mobile resources, I wish I could give my clients a short contract to sign promising that if they work with me, they will definitely incorporate usability into their project. They will listen to my suggestions and they will follow through with all the recommendations offered that make their product easier for users to use.

I, the ever grateful client (customer, boss, coworker), promise to incorporate usability into my project, to listen to all of your suggestions, to take your recommendations, and to follow everything that you have to say in order to improve usability and provide for the betterment of humankind.

In reality, if I were to actually suggest this, I would not have much success but rather would be seen as a crazily idealistic usability specialist. Likely, I wouldn’t even get to work on the project at all.

The truth is that, in many cases, user research is not a project unto itself. Rather, it is part of a larger Web-centric project effort. These larger projects often have a project manager or a project management team. This person or team has the task of coordinating the usability effort into the broader project launch schedule with the ultimate approval of the primary stakeholder or stakeholders.

The project manager or management team generally has some kind of technology management background, but often without extensive experience in usability. This lack of knowledge about usability isn’t inherently a problem, and this is the reason why my company is pulled into the project in the first place. However, a lack of knowledge about the value of usability research can bring about some reluctance. Based on my experiences and lessons learned, here is a "cheat sheet" of things the project manager or management team will often be considering when planning and incorporating user research into projects.

No matter what your role is on any given project, you can help make usability, and specifically user research, a priority. If you hear any of the counterpoints to usability that appear below, this cheat sheet will prepare you for how to respond to support important user experience research.

"I’ve got an incredible design team. I don’t need to have anyone evaluate it."

I never send out any kind of formal document without having one of my staff review it. No matter how good a writer I think I am, I am 100% sure that I’ll have typos in that document that I won’t find no matter how much I try to look for them. I’ll miss them because it’s my baby. Similarly, a design team, no matter how good they are, will not be able to see many issues with something they developed. A design team may also wish to do their own testing where the designers also operate as usability test moderators.

Recommendation: Don’t assume that a design team, no matter how amazing it is, will be able to see flaws in their creation. Although someone who is an interface designer can also be a usability evaluator, I’d highly recommend that they don’t evaluate a creation that they had a large part in designing. The risk of asking biased questions or having biased responses is much higher, and the risk of not seeing flaws is all that much greater. Ideally, someone who has had no role in the creation should be evaluating the site or application.

"Usability testing is too expensive."

I sometimes hear the contention that the budget doesn’t allow for usability testing. Once figuring in not only labor hours but also other direct expenses, such as participant recruiting costs, participant incentives, and facility rental, costs may seem like they are really starting to add up.

Recommendation: There are a number of arguments to be made for the cost-justification of usability testing. Even as early as the mid-1990s, there were books and articles published on this very topic (such as Cost-Justifying Usability, a classic revised in 2005). In addition, usability testing doesn’t have to be terribly expensive. First, calculate out what a typical usability study might cost, with an optimal number of representative users in an optimal location with optimal incentives. Look at the final number: is it really too high? If so, consider reducing the number of participants; even if not optimal, at least a fair bit of valuable information is likely to emerge from testing. Similarly, I like to produce full written reports with test results and recommendations, along with a PowerPoint summary report, but if budget (my allotted hours) doesn’t permit, I might just produce the PowerPoint summary report and make certain to at least convey the key ideas of the findings. In the more uncommon instance that the budget is larger than the base optimal projections, a usability researcher can always add what I’d consider "deluxe" options, such as a highlights video from the sessions.

"But we really can’t afford testing."

Sometimes the budget is really tight, and the client feels that they absolutely can’t do usability testing.

Recommendation: While not optimal, at a minimum you should consider a quick expert review. I’ve done reviews for clients with results given as short bullet points in an email, which helps save time and thus cut costs. Expert reviews never let me see a real user’s actions or hear their thoughts as they "think aloud," and I also must rely on whatever assumptions and knowledge I possess about users’ usage. So along with an expert review, certainly consider a usability test with only a small number of users that is observed by the stakeholder. Instead of spending time writing any report or even a summary, consider a quick debrief meeting a day or two later to talk about trends and observations. Again, this might not be optimal, but at least there is some evaluation time squeezed in.

"Our schedule is too tight for usability testing."

Often, timetables are tight and stakeholders may feel rushed. When stakeholders hear that a usability study may run over a period of three days and that the report will take a few more days, they may get the impression that the usability evaluator expects all work to stop during this period. Stakeholders will express the concern to the project managers that there simply isn’t enough time for usability testing.

Recommendation: Let the stakeholder know that not only do you know that time is tight, but you also know that the usability practitioner does as well. Other work on the project does not have to stop for the usability test to take place. Further, if the development environment is flexible enough, and ideally if there is perhaps an extra day or two between sets of participants, there may be enough time not only to make changes to a prototype, but also to test out some of the changes with the next round of participants or to adjust the test plan and drill down and zero in on key problems very rapidly.

"Usability testing will increase my project scope dramatically. Even worse, I don’t want to have to start from scratch!"

There may be a looming deadline, and even if there is the time to squeeze in a usability test, stakeholders may be afraid that the results are going to be less than perfect. They’re envisioning at least one whopper of an issue and don’t want to have to delay the deadline in order to get that issue resolved. Even scarier, they may believe that inherent in any usability test comes a risk that they will be told that they should be starting from scratch.

Recommendation: In my experience, with a good development team that has a reasonably good sense of design principles, the testing often uncovers a few larger changes, but usually it uncovers a number of problems that can be fixed with minimal impact to the development schedule. Even if the stakeholder sees some large-scale recommendations, assuming that there are no show-stoppers, these should be cataloged and saved for a future release. It’s okay to start your list of fixes for the next release even before this one is out. All the minor tweaks and low-hanging fruit, however, could be attended to immediately, and the stakeholder could see a large value-add for minimal cost.

"We’re already doing software testing—we’ve got it covered!"

Sometimes project managers are aware that there is this testing approach called "usability testing," but they figure it’s not much different than the software/functional testing that they are already doing. They may have spreadsheets of actions that a user is supposed to be able to do, and they need to make sure that all of the design specifications are met. While this software testing is certainly important, project managers may not realize that usability testing is focused on the success of actual users using the website, instead of just verifying that the functionality exists and is working as initially intended. What if those initial intentions and assumptions were incorrect for the expected audience group?

Recommendation: Consider the needs of the actual users of the site, not just the specifications that have been handed down to you by the stakeholders.

"I’ve done testing for my desktop Web projects, but do I need to do testing for similar mobile content/apps as well?"

Suppose you have done a good job integrating testing into websites in the past. But now, you are doing a mobile project. Maybe it’s your first. Maybe you’re "converting" your desktop site to a mobile site, or maybe you’re porting functionality to a new mobile app, or maybe you are creating a "responsive design" that, in theory, should be completely usable across devices. Since things worked smoothly on the desktop version, do you need to concern yourself with mobile version testing too?

Recommendation: Absolutely. Usability is very much framed by context and audience. The context of desktop use is likely to be different than the mobile context. Users will probably be in different situations when they use the mobile version than the desktop version. Also, the users of each are apt to be different and only some of your desktop users are likely to be using mobile. It’s important to categorically understand these differences, which will inform both who you will be recruiting for your test and also what scenarios you might be testing with.

"I really do want to do usability testing, but the stakeholder is only permitting me one shot at it."

A stakeholder may concede that usability testing is important but say that only one round of testing will be supported. At what point should the testing be done? At the beginning of development to get things off on the right track? Late in the development cycle? Right before launch to confirm things work?

Recommendation: If you only have one shot at testing, the best time to do testing is at the latest point you can where changes to the website or application would not be very problematic. At a minimum, I’d suggest that there be click-through prototypes, but these do not necessarily need to have the actual polished look of the finished product. That said, if you have the clearance to test with enough (perhaps 10 or 12) representative users, consider spreading these users out, maybe half at the prototype stage and half once everything is more developed. Perhaps you could even divide your users over three points. If you can do usability testing at multiple points, also consider doing walkthroughs of design comps. You can then fix potential points of confusion before coding even begins.

Usability vs. the Three Constraints of Project Management

The project management triangle has three constraints: time, cost, and scope. It is important for the project manager and those on the project to understand how usability relates to these three constraints.

Time: On the one hand, introducing usability evaluation would seem to introduce more time into the effort. On the other hand, just like spending time defining good requirements can improve the ability to stay on track later on, so too can usability testing make sure that things stay on schedule by reducing back-tracking. If usability testing is conducted at several points along the development timeline, the possibility of finding major issues right before launch can be avoided.

Cost: Any cost for usability evaluation would seem to make the overall project cost go up. On the other hand, though, like time, the cost is likely justified by the prevention of the need to spend labor hours later on fixing what turns out to otherwise not be usable.

Scope: Usability should be part of an initial project scope, and it would be wise for project planners to assume that usability testing will find some things that need to be changed. If the changes required are too significant, and the project deadlines can’t be met if certain recommendations are taken, then as suggested, it would be better to place these (theoretically significant) recommendations into a future release than to avoid doing the usability testing at all.

Conclusion: Get Buy-in

Buy-in for usability activities is critical for any project. Considering usability from the outset and including this into any project will hopefully show a project stakeholder and perhaps even the project manager or management team that you are considering your end users and value project quality.

If someone questions the usability evaluation efforts, be prepared to explain the decision and defend this approach. Try to help this person or people to understand that including a usability component doesn’t indicate a lack of trust in the team, but rather is an appropriate step for any project.

Also, remember that one of the best things that you can do is make sure that the stakeholders are part of the usability activities to whatever extent you can convince them to participate. When I looked in my photo archive of some of my own studies, it confirmed my observation that sometimes stakeholders are not fully involved in the usability testing effort. While I couldn’t find a picture of a stakeholder observing a usability test through the one-way glass, I did find evidence of the stakeholder’s presence in the picture above—a coffee cup! When stakeholders observe usability testing, they get a great first-hand look at the usability of their product. When they observe real people stumbling and uncovering issues on their site, they are much more likely to realize the value of user research.

Cory Lebson has functioned as a user experience consultant since 1994. He established Lebsontech LLC (www.lebsontech.com) in 1997, and, with a staff of three, his core client base has included a number of organizations and has led to usability projects for government, nonprofit, e-commerce, and educational institutions.

Cory enjoys functioning as an evangelist for usability. His primary passion is for user research, and he has conducted hundreds of usability test sessions and user research activities. Cory also greatly enjoys teaching topics related to user experience and technology. He currently conducts monthly seminars and is a frequent speaker at local area events and national conferences.

Cory has an MBA focusing in marketing and technology management, as well as an MA in sociology and a BS in psychology. Cory is also the president of the DC chapter of the User Experience Professionals’ Association (UXPA) and is on the board of directors of UXPA International, as the director of strategic partnerships. Cory can be found on Twitter (http://twitter.com/corylebson) and LinkedIn (http://linkedin.com/in/lebson).