By Brian Still | Member
This column examines the ways technical communicators contribute to the development of more usable products, especially those used in complex, dynamic environments. Novel usability evaluation methods and design techniques, as well as those rediscovered or repurposed, will be the focus. Please send your comments, questions, and suggestions for future articles to me at brian.still@ttu.edu.
I was fortunate to attend a talk in the latter part of May where one of the leaders of User Experience (UX) thought, Dr. Tharon Howard, taught a master class examining the differences between our usability approaches in the past and our UX approaches now. Whereas we once adopted an accommodationist approach to researching users, we are now in a position where we are attempting less to make products work for users and more construct a situation that better “allows them,” according to Howard, “to successfully experience the interface.”
For those of us who have employed usability testing in the traditional way Howard describes as accommodating, it has in many ways often felt like a zero-sum game. Our relentless focus on finding and fixing bugs, although retaining some value, has become less satisfying for us and for those we are providing solutions. As people like Ginny Redish, Mike Albers, and Howard have been telling us for a while, it is increasingly the case that complex systems, which really describe almost any mobile interface or application we make and use, resist simple fixes. One set of bugs found and fixed often give way to new sets of bugs found and fixed. And one person’s bug isn’t necessarily another’s. Even researchers, which Rolf Molich demonstrated in an insightful 2004 comparative study, often don’t find the same problems or recommend the same solutions. Maybe this is why Lane Becker didn’t hesitate at roughly the same time in calling most usability testing worthless.
Of course, it isn’t worthless to test the usability of a product, especially if the alternative is not to test it and then just hope everything will be alright. At the same time, to understand the user experience (UX) is not the same thing as to test for product usability. So much more goes into UX, and over the last decade or so many of us have committed more time to mapping the user experience, either in an effort to understand what might be ineffective about an existing product, or to design one from its very inception to be effective. Rather than seeing a product as an unfinished sculpture that needs final chiseling to remove blemishes and give it a perceivable shape, those of us interested in the user experience are seeing the product as part of that experience. We cannot focus solely on the product’s shape when the knowledge, expectations, and experiences—the mental model—of the user are the prism through which they see and use the product.
We don’t just want to ask, “can a user do something with a product?” As soon as we fix the product to let them do that task, there will be something else they will want to do, or the environment of use will change how they can do it. Instead, we want to know the following:
- How do users think and interact with a product?
- What do they require?
- What have others done before, both bad and good, to engage users?
- What does the company want to accomplish with the product they are making for the intended users?
UX is the whole onion, layer inside of layer. As this useful diagram below details (referenced from a 2008 nnGroup Amsterdam conference presentation), UX is, in fact, usability, utility, brand, and so many other things that constitute the dynamic environment where the user interacts with the product (see Figure 1).
The problem, and one Howard noted in his presentation, is that we’re short on research methodologies to make UX feel as effective as traditional usability. Yes, we can always iterate testing, but often clients only want or only give us enough time for one test. Lean UX, as I noted in a previous Intercom column, shows promise because it allows for short, hypothetically-based testing meant to construct insights into UX, especially in Agile development frameworks that have rapid sprints (two weeks or less) from one product creation stage to the next. Howard emphasized the value of user journey maps as comprehensive first efforts, and I’ve also highlighted their capabilities (also in a previous Intercom column). But since I have previously discussed journey maps, I wanted to use this column to touch on a couple of other methods you can make use of when attempting to construct a more useful understanding of UX: site visits and user shadowing.
Site Visits
Carol Barnum writes that to learn about users, “you must first observe users, then talk to them, and match what they tell you with what you see and comprehend.” The site visit offers an ideal way to accomplish this, allowing you to interact with users carrying out real tasks in real situations. St. Mary’s University Press in Winona, Minnesota, depends so much on the information gained from site visits that it requires all of its customer-contact employees, from those taking orders on the phone to executives, to carry out up to ten site visits a year.
Of all the methods aside from actual user testing, site visits are the most intensive, perhaps also the most expensive, but they are also typically the most rewarding. Your sense of the use environment can also be measured against what it’s really like.
So how does the site visit work? There are many different approaches, and you can limit the time to a couple of hours (you might hear this called a discount user observation), or you can elect to carry it out over a period of days. You literally want to go on site and observe users in action, doing what they typically do, how they typically do it, and where they typically do it. One of my most informative site visits was when I conducted a ride along with a state trooper a few years ago. I was consulting for a software company interested in creating a new app for law enforcement. To understand how it would be used and thus designed, I needed to know in what context it would be used. So I rode along with the trooper for a number of hours. I never could have recommended to the company what needed to be included in the design of the software to account for all of the distractions the trooper faced, such as the need for audio to impart messages as opposed to long text phrases that are difficult to read when driving and observing.
Be prepared to take notes, collect audio and video if possible (photos will work as well), make drawings if need be, and even bring back copies of artifacts, such as tools, books, or other resources that the users make use of while they work. You will want to ask them questions. Every good site visit offers a collection of triangulated data—user verbal feedback, your observations, and performative data, such as the actions or tasks they carry out. If users cannot speak to you while they work because what they are doing would be distracted by talking to you, ask questions after the fact or retrospectively.
If we plan to recruit users for a user test, a good site visit or series of site visits form the basis of user profiles we recruit from, as well as tell us what kinds of tasks and scenarios we should use in testing. Site visits also help us as we design new products because we understand better how users actually use the existing product or similar ones. Ultimately, a good site visit collects three things:
- User Analysis—knowledge about what the user does, thinks, and says.
- Task Analysis—observations of every task and subtask performed.
- Environment Analysis—documentation of every environmental thing—noise levels, light, size of area, objects in the area, resources used for support, and interruptions that occur. It wouldn’t hurt to bring back artifacts of things the user relies upon or uses (take photos of these if necessary).
User Shadowing
The user shadowing method falls somewhere during the site visit, when you are actively engaged with the user as a non-intrusive observer. A lot of method actors in preparation for movies carry out user shadowing. If you want to play a cop realistically in a movie, you act like the cop, imitating the cop’s actions, language, and other key activities. The idea then is that when it comes time to act in the movie, your mannerisms and words will be more realistic.
We use shadowing when we want to try to understand how users are thinking as they do something. And rather than engage them, say via a site visit, in a way that might be distracting, we elect instead to follow them around and do what they do as much as possible.
Obviously, the users you are shadowing need to know you’re doing this because it could be awkward if they didn’t. If they do, however, shadowing can be a productive tool, especially in the earliest stages of the design process, helping you get insight on a specific task or activity.
After focus groups, surveys, interviews, and other pre-field research about the product, users, and environment, you’ll most likely have identified your product’s target users and target environment. This is important because not only do you want to shadow the right users in the right location, you also want to have some idea of why you’re doing it. Do you have a hypothesis about how the product will be used and you want to test that out? The better focus you have will make the shadowing productive.
It may be that site visits and user shadowing don’t necessarily offer the sort of comforting quantitative data that we’ve come to expect from particular usability methods. Still, if we are interested in understanding the entire context of UX in order to build, evaluate, and then monitor and maintain an effective and satisfying, user-centered product, site visits and user shadowing allow for that, can be done alone or in groups, and are great tools for getting you outside of the building.
Reference
Barnum, C. M. 2010. Usability Testing Essentials: Ready, Set, … Test. Elsevier.
