Nicole Loorbach, Joyce Karreman, and Michaël Steehouder
Abstract
Purpose: Research shows that confidence-increasing elements in user instructions positively influence senior users’ task performance and motivation. We added verification steps and personal stories to user instructions for a cell phone, to find out how seniors (between 60 and 70 years) perceive these confidence-increasing motivational elements. Verification steps allow users to check if a procedure was carried out correctly. Personal stories describe how a fictitious user succeeded in completing a procedure.
Method: The plus-minus method was used. Participants were asked to place pluses and minuses in cell phone user instructions to indicate their positive and negative reading experiences, and to explain their choices afterwards.
Results: Verification steps are regarded as other, common parts of user instructions by seniors. Nineteen out of 20 participants appreciate verification steps, and they mainly encourage providing these because they are expected to decrease doubts, and increase user confidence and satisfaction. Personal stories tend to stand out compared to other, common parts of user instructions. Thirteen out of 20 participants appreciate personal stories, which use is mainly discouraged for redundancy reasons, and encouraged for their expected positive effects on user confidence and motivation.
Conclusion: The use of verification steps and personal stories in user instructions seems acceptable to seniors. As a next step, it seems worth it to test for effects of these motivational elements on usability, user confidence and motivation.
Keywords: plus-minus method, motivation, confidence, seniors, user instructions
Practitioner’s Takeaway
- Verification steps in user instructions are appreciated because they are expected to decrease doubts, and increase confidence and satisfaction.
- When seniors discourage the use of personal stories in user instructions, they mainly do so for redundancy reasons. When they encourage their use, they mainly do so for their expected positive effects on user confidence and motivation.
- Both verification steps and personal stories can be relatively easily added to existing user instructions; modifications to the existing instructions are not necessary. When offering these motivational elements in user instructions, technical communicators should present the elements distinctively, making it easy for users to skip them if they wish to.
Introduction
For a long time, user instructions were considered as purely instrumental documents: Instructions had to enable readers to perform tasks. And even though this still remains the main purpose of user instructions, views on how to accomplish this have changed over the years. The traditional view seemed to assume that when the instructions were correct, readers would automatically be able to perform the described actions. In other words, instrumental discourse alone should be enough for readers to reach their goals. Or as Moore (1997) stated: “Instrumental discourse does not persuade like rhetoric; it shows a user how to perform an action. … Instrumental discourse does not necessarily use reasons or appeals to logic, to the author’s character, or to the audience’s emotions. … Rhetorical communications and salespeople may persuade customers to buy specific hardware and software, but after the sale, the customers require no persuading to read and apply the installation and operating instructions. External circumstances obligate them to perform those tasks so they can use their new purchases” (p. 166).
The rhetorical view on the design of technical documents still assumes that above all, instructions should enable readers to perform tasks with the accompanying device. But in order to accomplish this, the instructions should motivate readers to start reading the instructions and to keep on reading once they have started doing so. Goodwin (1991) argues that “Audience motivation … plays a vital role in effective technical communication. Specifically, a good technical writer must keep a reader reading long enough and carefully enough to become competent at specific tasks” (p. 100). Horton (1997) advocates a similar view, suggesting that technical documents should motivate readers, and that technical writers should take responsibility for making the reader notice, understand, and act on the information. Horton labels instructional documents that enable users to do and find as “friendly documents.” These friendly documents present information clearly, make a case, and are readable. Horton labels documents that are enhanced by motivational elements as “seductive documents.” Such documents impel readers to do: They show, teach, convince, and get read. According to Horton, “friendly documents allow access to information-if a reader is motivated and tries to find it. Seductive documents go further and supply the motivation” (p. 6).
Thus far, we have conducted two experiments studying the effects of motivational elements in user instructions (Loorbach, Karreman, & Steehouder, 2007; Loorbach, Steehouder, & Taal, 2006), which suggest that providing motivational elements in user instructions, and especially elements aimed at increasing user confidence, will be beneficial for users, and senior users in particular. The first study tested the combined effects of a variety of motivational elements in user instructions for a fixed, wireless phone, on usability aspects and students’ appreciation for the instructions. The results showed that motivational elements led to a higher satisfaction with the instructions, but these elements did not have any effect on task performance. The participants in our second study were senior users, aged between 60 and 70. We asked them to perform a number of tasks with a cell phone. We decided to focus on this particular user group, because this large user group is often ignored in technical communication research (Lippincott, 2004). It is known that seniors are willing to use technology, and are even eager to learn, but training times are typically longer for seniors, and they may require more practice and assistance during training (Czaja & Lee, 2007; Naumanen & Tukiainen, 2007; Tsai, Rogers, & Lee, 2012). Designing user instructions that stimulate users’ willingness and as such, their ability to perform the described actions is especially called for in greying societies, with increasing numbers of seniors who are less experienced with relatively new technology devices (Schwender & Köhler, 2006), and less likely than younger adults to use technology (Czaja et al., 2006). Providing motivational elements in user instructions may especially benefit seniors.
The results of the second study showed that providing confidence-increasing elements in user instructions not only improves usability aspects of the instructional text, but it also increases user motivation: Senior users not only performed better, but were also more motivated to keep trying in the face of difficulties when using instructions containing a combination of four confidence-increasing elements. These elements were based on Keller’s (1983, 1987a, 1987b, 1987c, 1999, 2010) ARCS Model of Motivational Design. The ARCS Model provides strategies focusing on four objectives—(A)ttention, (R)elevance, (C)onfidence, and (S)atisfaction—to make instructions motivational, in order to increase student motivation in learning and performance settings (Keller, 2010). As a next step, we will test whether applying individual confidence-increasing strategies to user instructions can produce similar effects. The current article describes seniors’ perceptions of individual confidence-increasing elements in cell phone user instructions.
Motivational Design of User Instructions
According to Keller (1987a), “the first step in design is to create a list of potential motivational strategies for each of the [motivational] objectives. … The next step is to critically review the potential strategies, and select the ones to be used” (p. 7). In our previous study (Loorbach, Karreman, & Steehouder, 2007), we created and tested for collective effects of three lists of four potential motivational strategies for the motivational objectives attention, relevance, and confidence. For example, to increase attention, we added colored headings and headings written as questions. To enhance perceived relevance, we added “What’s coming up” sections focusing on the usefulness of cell phone functions. And to enhance confidence, we added “What’s coming up” sections focusing on confidence aspects, and verification steps. The collective effects of the four confidence elements, and to a lesser extent of the four relevance elements, proved effective, so our next step is to critically review the effective strategies that were used, and select the ones to be tested individually.
Keller (1987a) provides five guidelines to help the selection process: “Motivational strategies should: (a) not take up too much instructional time, (b) not detract from the instructional objectives, (c) fall within the time and money constraints of the development and implementation aspects of the instruction, (d) be acceptable to the audience, and (e) be compatible with the delivery system, including the instructor’s personal style and preferences” (p. 7).
The primary goal of this study was to investigate if motivational strategies are acceptable to the target group of seniors (guideline (d)). The results of the previous study showed that motivational elements had collective positive effects on task performance, but it is still unknown if users accept individual motivational elements as valuable parts of user instructions.
In selecting strategies to test individually, we tried to meet the other four guidelines. Our potential strategies should not take up too much instructional time, and add to rather than detract from the instructional objectives (guidelines (a) and (b)).We only considered strategies that can be relatively easily applied throughout existing user instructions in order to make them motivational: By making it relatively easy for instructional designers to design motivational user instructions, chances increase they actually will (Keller’s guideline (c)). And by looking for strategies that are as non-similar as possible, varying preferences of both technical writers and users are served (guideline (e)).
Apart from Keller’s guidelines, we used Bandura’s concept of self-efficacy to select strategies (1986). Bandura defined self-efficacy as “people’s judgments of their capabilities to organize and execute courses of action required to attain designated types of performance” (1986, p. 391). So, a high level of self-efficacy means that users have confidence in their ability to perform a task correctly. Bandura described four sources of self-efficacy beliefs: “enactive mastery experiences,” “vicarious experiences,” “verbal persuasion,” and “physiological states” (Bandura, 1997). We aimed at selecting strategies that meet the criteria of the first or second source of self-efficacy beliefs, since those are expected to affect self-efficacy beliefs most effectively.
We ended up with providing verification steps and providing personal stories as motivational strategies to test. Verification steps are extra steps at the end of procedural lists, which allow users to check whether the described actions were performed correctly. Personal stories describe how using a feature of the cell phone was a challenge to a persona, but success prevailed. Both verification steps and personal stories can be ignored by users who do not need or wish to read them, and used by those who do.
Looking back at the two views on designing user instructions (where the traditional, instrumental view advocates concise instructions focusing on effectiveness and efficiency, and the rhetorical view advocates also focusing on user satisfaction and motivation), verification steps are closer to the traditional view on designing user instructions, and as such, they would probably still go well with people advocating merely telling-like-it-is procedural steps; they seem to comply with well-accepted procedural lists, being an extra step at the end of such lists. Personal stories are closer to the rhetorical view on designing user instructions: These are far from common in the field of technical communication, but may positively affect user satisfaction and motivation.
Verification Steps
In our previous study (Loorbach, Karreman, & Steehouder, 2007), providing verification steps was one of four strategies aimed at increasing confidence. These strategies had collective positive effects on seniors’ effectiveness in performing tasks, and on their persistence in trying to complete tasks (motivation). Verification steps are expected to stimulate what Bandura (1997) calls “enactive mastery experiences,” in his description of the four principal sources of self-efficacy beliefs (pp. 79-115). This first and most effective source of self-efficacy beliefs serves as an indicator of capability. According to Bandura (1997), “enactive mastery experiences are the most influential source of efficacy information because they provide the most authentic evidence of whether one can muster whatever it takes to succeed. Successes build a robust belief in one’s personal efficacy” (p. 80).
Keller (1983) also acknowledges the importance of successes in building self-efficacy, or in his words “expectancy” (which he later renamed “confidence”). In his explanation of the first strategy to increase expectancy (“Increase expectancy for success by increasing experience with success”), he states: “If a person has a generally low expectancy for success or a specific history of failure in a given area, then a series of meaningful successes in that area will improve the person’s expectancy for success” (p. 419).
Van der Meij and Gellevij (2004), although not using the same terms, also link feedback—such as the kind provided by verification steps—to user confidence, and to motivation, when discussing Gellevij’s (2002) dissertation on functions of screen captures in software manuals: “When users consult screen captures and discover that they are still on the right track, the pictures serve as positive feedback, which reinforces motivation. Especially for the novice user, this may be important to allay initial anxiety. Apart from checking progress, users can also use screen captures to verify whether the program has processed their input correctly” (p. 8). Verification steps are expected to serve in a comparable way.
By providing verification steps—where possible, meaning they are adding value—at the end of procedures, we hope to stimulate readers to actually experience successes, because these steps enable users to check if the described actions were performed correctly. Verification steps should first of all take away possible doubts about whether or not the described actions were performed correctly. If success was the case, then verification steps are a confirmation of indeed having succeeded. In the opposite situation, verification steps will inform users that the actions were not performed correctly and as such, will allow for a retry and indirect success after all.
Personal Stories
Even though personal stories were one of four relevance-increasing elements in our previous study, which collectively positively affected seniors’ effectiveness in performing tasks, we believe that personal stories can also be used as a confidence-increasing element. Bandura (1997) describes the second source of self-efficacy beliefs as “vicarious experiences that alter efficacy beliefs through transmission of competencies and comparison with the attainments of others” (p. 79). This source is better known as modeling. According to Bandura, “seeing or visualizing people similar to oneself perform successfully typically raises efficacy beliefs in observers that they themselves possess the capabilities to master comparable activities. They persuade themselves that if others can do it, they too have the capabilities to raise their performance” (p. 87).
So by focusing on how someone thought completing a procedure was somewhat of a challenge, but succeeded after all, personal stories should increase confidence. The nature of personal stories automatically calls for some focus on relevance, the second objective of the ARCS Model, as well: When someone describes his or her actions with a cell phone, the relevance of these actions to the user almost inevitably shines through. In our previous studies, personal stories focused on relevance, and on why a certain feature was useful to someone. This time around, the focus will lie on increasing confidence, and on stimulating users to feel that they, too, can perform certain tasks with the cell phone.
Research Questions
Our two strategies fall into Bandura’s description of the first and second sources of self-efficacy beliefs, and they seem to meet four out of Keller’s five selection criteria. In order to answer whether they also meet guideline (d)—“being acceptable to the audience”—more research is needed. We do know that our motivational elements, and especially personal stories, are not common in user instructions, but we do not know how seniors react to their presence in user instructions. Therefore, we set up a study to reveal seniors’ reactions to our intended motivational strategies in cell phone user instructions, using the plus-minus method (cf. De Jong & Schellens, 1998). The plus-minus method involves participants placing pluses and minuses at text parts, and explaining their choices afterwards. We wanted to find answers to three research questions:
- How do seniors regard verification steps and personal stories in user instructions? Do seniors score these motivational elements as they would other logical, accepted, and in their view common parts of user instructions, indicating that these strategies are, too? Or do they score these elements differently than other parts of user instructions, indicating that to seniors, these are not regarded logical, accepted, and common parts of user instructions? How verification steps and personal stories are regarded by seniors is measured by counting the pluses and minuses participants place at the motivational elements, and by comparing these to the pluses and minuses placed at other parts of our test materials.
- Do seniors appreciate the use of verification steps and personal stories in user instructions? Whether verification steps and personal stories are appreciated by seniors is measured by asking our participants whether they generally encourage or discourage the use of these motivational elements in user instructions.
- Why do seniors encourage / discourage the use of verification steps and personal stories in user instructions? We expect seniors to link their positive answers to user confidence and motivation aspects, since these are reasons advocates of motivational elements in user instructions encourage their use (cf. Goodwin, 1991; Horton, 1997). We expect seniors to link their negative answers to redundancy, since advocates of the traditional view on designing user instructions see this as the major reason to discourage the use of motivational elements (cf. Moore, 1997).
The plus-minus method allows for testing views on and appreciation for text parts without focusing on them; a necessity for finding out whether the motivational elements in our text stand out or if they are seen as a logical part of the user instructions.
Translation verification step (right page, step 8):
You can now check if you have assigned the speed dial key to the chosen phone number: The display shows Key [number speed dial key you have just assigned] and a name and phone number. If this contact is assigned to this key, then the display will also show Options.
Translation personal story (left page, next to picture):
Ria Damhuis (68) enjoys her daily walks with her dog Ranka to the fullest. Her rheumatoid arthritis makes it hard to walk long distances, but on good days, she doesn’t mind walking a bit further to visit her daughter and grandson. She’ll call first to make sure they are home, but her rheumatoid arthritis makes it hard sometimes to push the buttons on her cell phone. She has assigned the phone number of her daughter to speed dial key 2, so she only has to hold down that key to call her. It was quite a challenge to assign the speed dial key, but it worked! Now, she doesn’t have to push her daughter’s number in anymore or look it up in her contact list: she only has to hold down key 2 and her cell phone does the rest.
Method
Materials
Our test materials were a chapter in our rewritten and redesigned Nokia 1100 user instructions in Dutch. An earlier version of these instructions was used for our previous study (Loorbach, Karreman, & Steehouder, 2007). We used the chapter on speed dialing, which comprises four pages. This chapter begins with a chapter title, a general introduction, and two remarks. The rest of the chapter contains six subchapters on speed dialing, and an equal number of step-wise procedures. Each subchapter contains a title and an introduction, and one subchapter contains a remark. Throughout the text, three personal stories are included, on pages 1, 2, and 3. Personal stories are either anecdotes or testimonials, describing how a 68-year-old woman named Ria Damhuis struggled a bit with the instructions, but always succeeded in reaching her goal. Each story is accompanied by a picture of Ria to stimulate the process of modeling.
Also, four verification steps are included: once on pages 2 and 3, and twice on page 4. In total, our test materials consist of 7 titles, 7 intros, 3 remarks, 27 steps, 4 verification steps, and 3 personal stories. See Figure 1 for the first two pages of our chapter on speed dialing (in Dutch).
Participants and Procedure
Twenty Dutch seniors (6 males and 14 females; age range 62–70, M = 66.75, SD = 2.71) participated in our study. Participants either replied to an advertisement in newsletters of several elderly associations, to a flyer that was put in their mail box, or to a request of an acquaintance who had already participated in our study. Participants received €10 for their cooperation.
The only selection criteria we used were that the participants were between 60 and 70 years of age, and that they did not have any prior knowledge on how to use a Nokia 1100, the type of cell phone described in the test materials.
After having participants fill out some questionnaires on confidence aspects as a pre-test for our next study, we used the plus-minus method to invoke seniors’ responses to our test materials. De Jong and Schellens (1998) explain that the plus-minus method “involves asking participants to read a text from start to finish and to mark their positive and negative reading experiences with pluses and minuses, respectively, in the margin. Pluses and minuses may be assigned to all sorts of text elements (from chapters to words) and for various reasons (for example, comprehensibility, appreciation, relevance of the information). After that, individual interviews are held, focusing on the reasons for every plus and minus” (p. 123). This method is primarily a qualitative method, and even though it has been used quantitatively before, it has never been used to compare the number of pluses and minuses given to relatively new parts in user instructions to the number of pluses and minuses given to well-accepted, common parts in the genre of technical communication.
As suggested by De Jong and Schellens, we asked each participant to read the four-page chapter on speed dialing, and to mark their positive and negative reading experiences with pluses and minuses. In explaining the procedure to participants, no references were made to the motivational elements in the text. When participants were finished scoring the text, a voice recorder was started (all participants signed an approval form), and participants explained each plus and minus they had marked in an interview. At the end of each interview, the interviewer pointed out the verification steps and personal stories. When participants had not placed any plus or minus next to a verification step or personal story, or when they had placed both a plus and a minus, then the interviewer asked: “If you had to place one plus or minus, what would it be?” All participants were asked: “If it were up to you, would you encourage the use of verification steps / personal stories in user instructions, or would you discourage its use? And why?” As a result, all participants marked verification steps and personal stories, and explained their reasons for doing so.
Data Analysis
All pluses and minuses were explained during the interviews. All the pluses and minuses that were marked before the interview were labeled “initial pluses and minuses.” When participants’ plus or minus next to a verification step or personal story was placed as a response to the element in question, then we recorded this entry as a “spontaneous” plus or minus. A spontaneous plus or minus for verification steps means that the participant either liked or disliked the idea of providing verification steps in user instructions. For personal stories, pluses and minuses are either related to the idea of having personal stories in user instructions, or to Mrs. Damhuis, the character displayed in all personal stories. So when participants explained that they had given a plus because a procedure was explained clearly due to the verification step, because using speed dial is very useful to Mrs. Damhuis, or because of Mrs. Damhuis’ ability to use speed dial, then a spontaneous plus was indeed attributed to the verification step or personal story they had initially placed it at. But when participants explained that they had given a plus because using speed dial is very useful period, because they particularly liked the use of a certain word in a verification step, or that a minus was given because Mrs. Damhuis should have purchased a customized cell phone for people suffering from rheumatoid arthritis instead of using speed dial on the described cell phone, then the plus or minus was categorized as “other,” and participants were subsequently asked to place and explain a plus or minus for the strategy per se; for the idea of providing verification steps or personal stories in user instructions. The resulting pluses and minuses were labeled as “forced.”
Participants who had not placed any plus or minus next to a verification step or personal story, or who had placed both a plus and a minus were also asked to place and explain one plus or minus for each strategy per se. The resulting pluses and minuses were again labeled as “forced.”
So, initial pluses and minuses are all pluses and minuses given by senior participants to text parts in the test materials. These pluses and minuses were not necessarily aimed at the motivational elements verification steps and personal stories. All final pluses and minuses were either spontaneously given or “forced” (given when asked by the researcher), and reflect seniors’ views on the idea of providing verification steps and personal stories in user instructions. The final pluses and minuses are two per participant; one for each of the two strategies per se.
The interviews on seniors’ reasons for encouraging or discouraging verification steps and personal stories in user instructions were coded by two raters. Cohen’s Kappa interrater agreement coefficient was calculated for their categorization of seniors’ explanations, after which disagreements were solved by reevaluating the accompanying explanation together and assigning it to the most suitable category.
Results
How Do Seniors Regard Verification Steps and Personal Stories?
We set out to test whether the confidence-increasing elements verification steps and personal stories are acceptable to seniors. As a first step, we wanted to find out whether these motivational elements are regarded by seniors as a natural part of user instructions: Whether they are treated like common and well-accepted parts of user instructions. We assume that if these motivational elements stand out, then they will be given a relatively higher number of initial pluses and minuses than other parts of the instructional text. In other words, if these elements are seen as other well-known, accepted parts of the user instructions, then our participants will treat them the same way, and give them a proportionally comparable number of pluses and minuses compared to the rest of the instructional text.
In total, verification steps were given 11 initial pluses and 7 initial minuses, meaning these pluses and minuses were not necessarily given to the use of verification steps in user instructions (that is, they could have been given, for example, because a certain word in the verification step was (un)appealing to a participant). Our test materials consisted of 51 parts, and contained 4 verification steps, and 27 regular steps. Assuming verification steps do not stand out, we expected them to be given a proportionally comparable number of initial pluses and minuses compared to regular steps, which we consider comparable, accepted parts of the instructional text. Our suspicion was confirmed by a Pearson Chi-Square test (χ2 = .307, p = .580): Seniors’ scoring of verification steps does not statistically differ from their scoring of other, regular steps (see Table 1), so seniors regard verification steps as a common part of user instructions.
In total, personal stories were given 11 initial pluses and 9 initial minuses, meaning they were not necessarily given to the use of personal stories in user instructions. Our test materials consisted of 51 parts, of which 20, like personal stories, were not steps. Our instructions contained three personal stories. Assuming personal stories do not stand out, we expected them to be given a proportionally comparable number of initial pluses and minuses given to other non-steps. Even though personal stories were given more pluses than expected, and fewer minuses than expected, and other non-steps were given fewer pluses than expected and more minuses than expected assuming the two categories are comparable (see Table 2), these differences merely tended to statistically differ (χ2 = 3.338, p = .068). So contrary to our expectations, personal stories merely tend to stand out, compared to common, well-accepted parts of user instructions.
Do Seniors Appreciate Verification Steps and Personal Stories?
To find out whether verification steps and personal stories are appreciated by seniors, we need to look at only those pluses and minuses that were given to the strategies themselves (to the idea of providing verification steps and personal stories in user instructions). Do seniors suggest verification steps and personal stories should be incorporated in user instructions “in real life,” or do they suggest these elements should be left out?
We looked at the final pluses and minuses given to verification steps and personal stories per se, meaning one plus or minus per strategy per participant, reflecting whether participants encourage or discourage its use in user instructions (see Table 3).
For verification steps, none of the initially given pluses and minuses regarded the strategy itself. This confirms our previous finding that verification steps are viewed as logical parts of user instructions by seniors. When asked, 19 out of 20 seniors encouraged the use of verification steps in user instructions. Or in Keller’s (1987a) terms: To 95 percent of our senior participants, providing verification steps in user instructions is acceptable.
For personal stories, 5 out of the 11 initially given pluses, and 3 out of the 9 initially given minuses regarded the strategy itself. These were labeled “spontaneous pluses and minuses.” This confirms our previous finding that personal stories tend to stand out to seniors: In our case, personal stories stand out positively to 25 percent, and negatively to 15 percent of seniors. Looking at the final pluses and minuses given to personal stories per se shows that 13 out of 20 seniors encourage the use of personal stories in user instructions. Or in Keller’s (1987a) terms: To 65 percent of our senior participants, a slight majority, providing personal stories in user instructions is acceptable.
Since all personal stories presented a female user, we were interested whether gender plays a statistically significant role in user perception. This was not the case.
Why Do Seniors Encourage / Discourage the Use of Verification Steps and Personal Stories in User Instructions?
Verification Steps. When asked, 19 out of 20 participants responded positively to verification steps: If seniors had to choose between encouraging or discouraging their use in user instructions, then 95 percent would encourage it. The one person who discouraged their use explained that people will be able to tell if the described actions were performed correctly, and therefore do not need verification steps. As predicted by Moore (1997), who opposes to the idea of providing motivation in user instructions, the use of this motivational element is discouraged because it is considered redundant.
The interviews on the reasons for seniors’ final pluses and minuses reveal that, as expected, verification steps are mainly encouraged because they are expected to decrease possible doubts, and to increase user confidence and satisfaction. Following are some examples of seniors’ explanation for encouraging verification steps:
Table 4 shows a categorization of seniors’ main reasons for encouraging verification steps in user instructions (Cohen’s kappa interrater agreement coefficient = .81).
Personal Stories. Thirteen out of 20 participants responded positively to personal stories: If seniors had to choose between encouraging or discouraging their use in user instructions, then 65 percent would encourage it. The interviews on the reasons for seniors’ final pluses and minuses reveal that, as expected, personal stories are mainly encouraged for their expected positive effects on user confidence and motivation. Following are some examples of seniors’ explanation for encouraging personal stories in user instructions:
[Personal stories] make it easy for us. Because we read this story about this lady who walks her dog and she wants to go visit, right? [“Yes”] And then I’m thinking “Well, that lady actually has the same problem we do.” Right? Because it says here “with effort, she managed to finish up,” and she was proud of that, too, towards the children, so to say. … When there’s a short story, it gives me a secure feeling.
I’m a bit in the middle there, because on one hand, it leads you to “Hey, oh, that’s a situation I find myself in, too, sometimes”. That’s useful and then you think “Well, if she thinks that’s very… then I should have a look after all.” Right? So I actually think, yes, an instruction manual, when I think of that, I don’t think of stories about people, but on the other hand, I do think it’s clarifying, especially for some people who aren’t just focused on technical things.
That speed dial key, that’s a plus, that it’s so easy for her to reach her daughter, I really thought that’s a plus. … This story is also positive. Yes. That she thinks “I’ll do that, no problem,” and when she actually started doing it, she thought “Oops, how should I do that?” And she managed after all, I think that’s very positive. Right? I’d try it, too. I’d really try it and think “Well, I’m going for it. I’m just going to try and if I get stuck, well, I’ll call André. [“He’s your personal helpdesk?”] Yes, my personal helpdesk. [But you would encourage stories like this?”] Yes. I would, yes. I would, because it tells you that many people are dealing with this. Right? And that many people are sometimes struggling with this. And I just think that’s great. Yes, really. … And to me, this story is truly amazing, that he’s keeping scores of the weather and that she sends him text messages. I think that’s truly amazing, when you’re able to do that, respect. Really. I can’t, I can’t send text messages.
Table 5 shows a categorization of seniors’ main reasons for encouraging personal stories in user instructions (Cohen’s kappa interrater agreement coefficient = .81).
As expected, and comparable to seniors’ reactions to verification steps, the use of personal stories is mainly discouraged for redundancy reasons. Following are some examples of seniors’ explanation for discouraging personal stories in user instructions:
That isn’t really part of it, or is it? That lady. Is it? [“Yes, that lady is part of the text”] … I don’t think it adds anything. … Someone else might… But I don’t really need it. No. Why this is here, this entire story about this woman, I’m not a fan of that. I just want to know what to do. … I put a minus, because I think it doesn’t make sense. Why should I know what that woman is doing? No, I don’t think it’s necessary at all. It really doesn’t have to be there. Just tell me like it is, dryly, well, clearly. And then that woman doesn’t have to be there, because I’m not interested in knowing what she’s doing with that thing.
Table 6 shows a categorization of seniors’ main reasons for discouraging personal stories in user instructions (full agreement between the two raters; Cohen’s kappa = 1).
Conclusion and Discussion
Value of the Plus-Minus Method
The plus-minus method proved to be a valuable method for checking if verification steps and personal stories are evaluated in a comparable manner to other, well-accepted parts of the instructional test materials. Another advantage of the plus-minus method is that it enabled us to find out how seniors score certain parts of the text—in our case, by looking at the pluses and minuses given to motivational elements—without putting a focus on those elements. In our view, this method is less prone to socially desirable answers, and therefore gives a more natural look at seniors’ true views on the use of verification steps and personal stories in user instructions than by asking them directly or by letting them fill out a questionnaire.
Seniors’ Reactions to Verification Steps
Our study showed that seniors regard verification steps as a natural part of user instructions: In the process of scoring instructional parts, verification steps were treated like other steps which are assumed a common, well-accepted part of user instructions. Besides being scored like other, well-accepted steps in user instructions, none of the responses to verification steps per se were spontaneously given. This means that all initially given pluses and minuses regarded other things than the strategy (for example, words used, presented order in the chapter, etc.). The fact that no spontaneous minuses were given to verification steps per se indicates that providing this motivational element in user instructions will probably not offend or annoy senior users.
The finding that verification steps do not particularly stand out in user instructions was expected, since they belong to “actions and reactions,” which according to Van der Meij and Gellevij (2004), are one of the four components of a procedure. Verification steps can also be seen as an elaborated kind of feedback statements in steps, which in turn are considered an occasionally used, but common part of user instructions (see Farkas, 1999, describing streamlined-step procedures, “a model that dominates online help systems and is very widely used elsewhere” (p. 42)). Where feedback statements in steps are “brief descriptions of the system’s response to the user’s action and the new state the system has entered” (p. 49), verification steps literally go one step further and provide feedback on the entire procedure. The purpose of both kinds of steps seems the same: “The most basic role of feedback is to provide verification (make clear that the user did the right thing and the system has responded properly) and to draw the user’s attention to the result of the action” (Farkas, 1999, p. 49). As we do with verification steps, Farkas believes feedback statements will positively influence user confidence: “Even though writers may not use feedback statements often, their deft use helps guide the user smoothly and confidently through the procedure” (p. 49). So verification steps are considered an accepted part of user instructions to our audience of seniors. They’re expected to increase user confidence and they may have a positive effect on usability.
Aside from seeming well-accepted to senior users, verification steps also seem to be appreciated by our audience: 19 out of 20 seniors encouraged providing them in user instructions. The one person who discouraged its use did so for redundancy reasons. Still, 95 percent of our participant group encouraged providing verification steps in user instructions, and we therefore consider verification steps an appreciated motivational element by our senior audience.
Our study showed that out of the 19 participants who encouraged the use of verification steps in user instructions, 13 did so because these are expected to decrease possible doubts, and increase user confidence and satisfaction. This is in line with our predictions, and with the reasons motivational elements in user instructions are encouraged by researchers in the field of technical communication, like Goodwin (1991) and Horton (1997). It is also in line with Bandura’s first source of self-efficacy beliefs, since decreasing possible doubts will stimulate experiencing success, and as such enactive mastery experience. One participant specifically named effectiveness of task performance—another stimulant of enactive mastery experience—when explaining why the use of verification steps in user instructions should be encouraged. So when looking at seniors’ reasons to encourage the use of verification steps in user instructions through Bandura’s eyes, so to speak, then 14 out of 19 participants (74%) did so because they expected them to positively affect enactive mastery experiences with success, which in turn will stimulate self-efficacy beliefs. We suggest providing verification steps in user instructions.
Seniors’ Reactions to Personal Stories
As far as personal stories are concerned as a motivational element in user instructions, we found out that they are less common in user instructions than verification steps: To our group of senior users, personal stories tend to stand out in user instructions. This was to be expected, since personal stories are not commonly used in user instructions (yet). Like with verification steps, we also looked at the number of spontaneously given pluses and minuses to providing personal stories as a strategy per se: Forty percent of pluses and minuses given to the strategy per se were spontaneously given, leading us to conclude that personal stories stand out more to seniors than verification steps do, which were not given any spontaneous pluses or minuses for the strategy per se. This is not surprising, since each story is accompanied by a picture, and opposed to verification steps which are a different kind but still another step that seniors are used to seeing in user instructions, anecdotes and testimonial are not commonly used in the field of technical communication. We expect personal stories to be regarded more like common parts of user instructions once they are being provided more often in this genre.
Even though seniors do not regard personal stories as a common, well-accepted part of user instructions yet, they do tend to appreciate the use of personal stories in user instructions: Thirteen out of our 20 senior participants, a slight majority, encourage providing personal stories in user instructions. Out of the 13 participants encouraging the use of personal stories, 5 did so for reasons in line with the rhetorical view on designing technical documents (cf. Goodwin (1991) and Horton (1997)), and in line with Bandura’s second source of self-efficacy beliefs called “vicarious experiences,” namely because personal stories are expected to install vicarious pride and as such, to increase user confidence and motivation, and because they are expected to stimulate modeling. So when looking at seniors’ reasons for encouraging the use of personal stories in user instructions through Bandura’s eyes, so to speak, then 5 out of 13 participants (38%) did so because they expected them to positively affect vicarious experiences with success, which in turn will stimulate self-efficacy beliefs.
Two out of the 13 seniors who encouraged the use of personal stories in user instructions did so for the sake of relevance. Even though we did not aim at increasing relevance, like we did when we designed the personal stories in our previous study (Loorbach, Karreman, & Steehouder, 2007), it is understandable that to our participants, relevance is a reason to encourage personal stories in user instructions. The nature of personal stories makes it impossible to solely focus on confidence, but that is welcomed rather than unwanted. Visser (1998) acknowledges that “the four dimensions of the ARCS Model are more to be seen as a four-dimensional look at motivation and thus cannot be strictly separated; often indeed the categories go smoothly over from one dimension into the other” (p. 145). In the process of narrowing down the number of strategies to design motivational elements in user instructions, we did not necessarily seek out strategies that strictly influence user confidence in using the instructions: We were looking for strategies that work well when it comes to increasing user confidence. From that perspective, automatically focusing on relevance as well is a welcome addition as opposed to a nuisance. Furthermore, our validation study of Keller’s (2010) Instructional Motivational Materials Survey (IMMS) (Loorbach, Peters, Karreman, & Steehouder, submitted) showed a significant direct effect of the relevance construct on the confidence construct, suggesting that increasing relevance will indirectly increase user confidence as well. As such, affecting relevance aside from user confidence is a welcome side-effect of providing personal stories in user instructions.
Out of the seven seniors explaining why they discourage the use of personal stories in user instructions, three also mentioned advantages of personal stories: “It allows you to imagine”, “What [Mrs. Damhuis] did and that she succeeded, that’s very nice, right? … I do want to know [Mrs. Damhuis] succeeded”, and “It’s very positive for [Mrs. Damhuis].” So three out of the seven seniors discouraging the use of personal stories do see and mention its benefits. This makes us believe that even when seniors do not want to read personal stories themselves, such stories, like expected of verification steps, will probably not offend or annoy senior users when they encounter them in user instructions, especially when skipping such stories is made relatively easy by offering them separately from the actual procedural instructions.
Six out of the seven seniors discouraging the use of personal stories in user instructions did so for redundancy reasons. In Horton’s (1997) words, these participants prefer “friendly documents” instead of “seductive documents.” But the participants in our study were asked to judge whether they encourage or discourage the use of motivational elements based on merely reading an instruction manual chapter, without having worked with the cell phone in question. Participants might have based their judgment on the assumption that the instructions will be clear and easy to follow, and everything will go according to plan; that their persistence will not be tested, and their confidence will not be dented. In other words, participants might have based their judgment on the assumption that motivational elements will not be needed. Diehl (2004) states that people evaluate a text differently when they merely read (cf. the described study in this paper) versus when they both read and do (cf. our planned study to test for effects of verification steps and personal stories on usability, user confidence and motivation). Our first study (Loorbach, Steehouder, & Taal, 2006) showed a similar trend: “After simply scanning the instructions and looking at the product, participants overestimated their [self-efficacy] responses, but after actually working with the text and the product to perform tasks, participants could gauge their responses more realistically. … Our study confirms that such judgments [based on merely reading the text] may not reflect the true comprehensibility or usability of these [instructional] documents” (p. 194). So even when seniors regard the use of personal stories as unnecessary or redundant, such stories might prove beneficial concerning confidence and motivation, and usability of the instructions, once they start working with the instructions and cell phone in question. We suggest providing personal stories in user instructions, bearing in mind that they are presented distinctively from the procedural instructions to allow for easy skipping.
Future Research
This study showed that the use of verification steps and personal stories in user instructions seems acceptable to seniors (cf. Keller’s (1987a) guideline (d)), provided that the design of the user instructions allows senior users to fairly easily skip the added motivational elements. Assuming that seniors will skip motivational elements when they realize they do not want to read them, and assuming the provided motivational elements are sufficiently delineated from other, essential parts of the instructions to allow for such easy skipping, we feel that verification steps and personal stories will probably not offend or annoy senior users when they encounter them in user instructions. We also predict that chances are slim these motivational elements will produce negative effects on usability and motivation aspects. Knowing this, it seems worth it to let seniors read and work with user instructions containing either verification steps or personal stories, and to test for effects of these types of motivational elements on usability, user confidence and motivation.
Acknowledgments
We thank three anonymous reviewers for their thorough review of our manuscript and for their helpful and thoughtful comments.
References
Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice Hall.
Bandura, A. (1997). Self-Efficacy: The Exercise of Control. New York, NY: W.H. Freeman.
Czaja, S. J., Charness, N., Fisk, A. D., Hertzog, C., Nair, S. N., Rogers, W., & Sharit, J. (2006). Factors predicting the use of technology: Findings from the center for research and education on aging and technology enhancement (CREATE). Psychology and Aging, 21, 333-352.
Czaja, S.J., & Lee, C.C. (2007). The impact of aging on access to technology. Universal Access in the Information Society, 5, 341-349.
De Jong, M., & Schellens, P. J. (1998). Formative evaluation in document design: Validity of the plus-minus method for the pretesting of public information brochures. In Proceedings of the 1998 International Professional Communication Conference (pp. 123-130). Quebec City, QU, Canada: IEEE.
Diehl, V. A. (2004) Access to affordances, development of situation models, and identification of procedural text problems. IEEE Transactions on Professional Communication, 47, 54-64.
Farkas, D. K. (1999). The logical and rhetorical construction of procedural discourse. Technical communication, 46, 42-54.
Gellevij, M. (2002). Visuals in Instruction: Functions of Screen Captures in Software Manuals. (Doctoral dissertation). University of Twente, Enschede, The Netherlands.
Goodwin, D. (1991). Emplotting the reader: Motivation and technical documentation. Journal of Technical Writing and Communication, 21, 99-115.
Horton, W. (1997). Secrets of user-seductive documents: Wooing and winning the reluctant reader. Arlington, VA: STC.
Keller, J. M. (1983). Motivational design of instruction. In C. M. Reigeluth (Ed.), Instructional-design theories and models: An overview of their current status (pp. 386-434). Hillsdale, NJ: Lawrence Erlbaum.
Keller, J. M. (1987a). Development and use of the ARCS Model of Motivational Design. Journal of Instructional Development, 10(3), 2-10.
Keller, J. M. (1987b). Strategies for stimulating the motivation to learn. Performance & Instruction 26(8), 1-7.
Keller, J. M. (1987c). The systematic process of motivational design. Performance & Instruction, 26(9), 1-8.
Keller, J. M. (1999). Using the ARCS motivational process in computer-based instruction and distance education. New Directions for Teaching and Learning 78, 39-47.
Keller, J. M. (2010). Motivational design for learning and performance: The ARCS model approach. New York, NY: Springer.
Lippincott, G. (2004). Grey matters: Where are the technical communicators in research and design for aging audiences? IEEE Transactions on Professional Communication, 47(3), 57-70.
Loorbach, N., Karreman, J., & Steehouder, M. (2007). Adding motivational elements to an instruction manual for seniors: Effects on usability and motivation. Technical Communication, 54, 343-358.
Loorbach, N., Peters, O., Karreman, J., & Steehouder, M. (submitted). Validation of the Instructional Materials Motivation Survey (IMMS) for seniors in a self-directed performance setting.
Loorbach, N., Steehouder, M., & Taal, E. (2006). The effects of motivational elements in user instructions. Journal of Business and Technical Communication, 20, 177-199.
Moore, P. (1997). Rhetorical vs. instrumental approaches to teaching technical communication. Technical communication, 44, 163-173.
Naumanen, M., & Tukiainen, M. (2007). Guiding the elderly into the use of computers and Internet—Lessons taught and learnt. In D. Kinshuk, D. G. Sampson, M. J. Spector, & P. Isaías (Eds.), Proceedings of Cognition and Exploratory Learning in Digital Age (pp. 19-27). Algarve, Portugal: IADIS.
Schwender, C., & Köhler, C. (2006). Introducing seniors to new media technology: New ways of thinking for a new target group. Technical Communication, 53, 464-470.
Tsai, W. -C., Rogers, W. A., & Lee, C. -F. (2012). Older adults’ motivations, patterns, and improvised strategies of using product manuals. International Journal of Design, 6(2), 55-65.
Van der Meij, H., & Gelllevij, M. (2004). The four components of a procedure. IEEE Transactions on Professional Communication, 47(1), 5-14.
Visser, A. J. M. (1998). The development of motivational communication in distance education support. (Doctoral dissertation). University of Twente, Enschede, The Netherlands.
About the Authors
Nicole Loorbach completed her PhD at the Department of Communication Science–Corporate and Marketing Communication at the University of Twente (The Netherlands). She is co-founder, research director and researcher at PURE Research, and founder of Duh txt, where she creates and transforms texts to communicate what matters most. She also teaches others to do the same, in scientific, corporate and personal communication. Contact: nicole@duhtxt.com.
Joyce Karreman is affiliated with the department of Communication Science–Corporate and Marketing Communication at the University of Twente. She teaches courses in document design and user support. Her research interests include the use and effects of different information types in instructive texts, document design for low literate people, and health communication. Contact: j.karreman@utwente.nl.
Michaël Steehouder is professor emeritus of technical communication at the University of Twente. He was chair of the Communication Department. His research interests include the design of public information and user support. Contact: mfsteehouder@gmail.com.
Manuscript received 16 November 2012; revised 22 July 2013; accepted 1 August 2013.
