Abstract
Purpose: To gather reliable empirical data on (1) STC members' use of and attitudes toward single sourcing and/or content management (SS/CM) methods and tools; (2) factors perceived to be driving or impeding adoption of this technology; (3) transition experiences of adopting work groups; (4) perceived impacts of SS/CM methods and tools on efficiency, usability, customer focus, and job stress.
Method: Cross-sectional sample survey of 1,000 STC members conducted in May 2008; multiple survey contacts by email with link to online survey instrument.
Results: Of 276 respondents, half reported using SS/CM methods and tools. About 1 in 10 respondents reported experience with a failed implementation of SS/CM; half the SS/CM users reported significant downsides or tradeoffs. Perceived top drivers of SS/CM adoption were faster development, lower costs, regulatory and compliance pressures, and translation needs. About 1 in 9 respondents used Darwin Information Typing Architecture (DITA). Large company size made use of SS/CM significantly more likely, and work groups using single sourcing with content management were significantly larger than work groups of other SS/CM subgroups and non-users of SS/CM. Single sourcing without content management seems destined to achieve a larger proportion of adopters than single sourcing with content management, barring a technology breakthrough. Among all respondents, Microsoft Word and FrameMaker were the most-used primary authoring tools.
Conclusions: With regard to these methods and tools, STC members appear to be in the Early Majority phase of Everett M. Rogers's innovation adoption curve. Diffusion of these methods and tools appeared to have been steady in the five years prior to the survey, with no dramatic increase in more recent pace of adoption.
Keywords: single sourcing, content management, methods and tools, technology transfer, survey methods
Practitioner's Takeaway
- Data from May 2008 show that single sourcing and content management were slowly and steadily being adopted by technical communication workgroups; however, these methods and tools were diverse, and no single kind of SS/CM method or tool seemed destined to become dominant.
- Single sourcing, both with and without content management, apparently had reached a critical mass of adopters, but content management without single sourcing had not.
- Microsoft Word and FrameMaker were respondents' most-used primary authoring tools, and more than three times as many respondents produced PDF files as produced content using the Extensible Markup Language or its predecessor, Standard Generalized Markup Language.
Introduction
During the past decade, scores of authors from both academic and practitioner ranks of technical communication have written and talked about methods and tools associated with the terms single sourcing and content management. Despite the steady flow of information and opinions on these topics (see Appendix A for a brief annotated bibliography), we have not had hard data on how many practitioners use such methods and tools and what they think about them. To fill that gap, we conducted a probability sample survey of STC members in May 2008.
We begin our report by defining key terms. In Objectives and Methodology, we state what we set out to learn, explain how we designed, tested, and deployed the survey, and describe how we analyzed the data. We organize the Summary of Results with statements summing up the most noteworthy findings that we took from the data, which we report in abbreviated form. In the Conclusions section, we recap and briefly discuss what the survey results tell us about STC members' use of single sourcing and content management.
Definitions Used in the Survey
Any discussion about single sourcing and content management should begin by defining those terms carefully. The terms are not synonymous, though often conflated, as anyone who researches these topics quickly discovers. Searching bibliographic databases or the Web using the term single sourcing, you may find case stories about single sourcing carried out using a content management system (e.g., Happonen & Purho, 2003; Petrie, 2007), but you may also find that a case is about an application or method that does not include a content management system (Welch & Beard, 2002). Likewise, results produced by the search term content management will list articles about a system that enables single sourcing (Hall, 2001; Pierce & Martin, 2004) as well as articles about a Web content management system lacking the functionality that would enable single-source publishing (McCarthy & Hart-Davidson, 2009; Pettit Jones, Mitchko, & Overcash, 2004). Indeed, many Web content management systems are designed in ways that make single sourcing impossible.
In our survey, we defined single sourcing by quoting a widely recognized authority on the topic (Ament, 2003). Kurt Ament defines single sourcing as
a method for systematically re-using information [in which] you develop modular content in one source document or database, then assemble the content into different document formats for different audiences and purposes (p. 3).
We want to emphasize that true single sourcing does not include cutting and pasting content from the source to different outputs; single sourcing uses software so that different outputs can easily be published from a single source document or database.
If we were to repeat the survey, we would revise Ament's definition to “you develop modular content in one source document, Help project, or database.” Widely used Help authoring tools such as Adobe Robohelp and MadCap Flare enable single sourcing as Ament defines it, but their primary content repository is a project, which is neither a document nor, strictly speaking, a database. A Help project collects and stores all the files needed to publish content, which can be customized for different audiences and products and/or different outputs, such as Web help and manuals in PDF (portable document format). Those who insist on absolute semantic precision with regard to this topic can expect to be frustrated for some time to come. The evolution of Help authoring applications like those mentioned (and others, no doubt) will even more thoroughly blur the distinction between single sourcing and content management.
We wanted our survey respondents to think of single sourcing as a method of information development distinct from content management systems, which we defined as a method-neutral technology:
For the purposes of this survey, content management systems are applications that usually work over a computer network and have one or more databases at their core; they store content, as whole documents and/or as textual and graphical components; they mediate the workflow to collect, manage, and publish content with such functions as maintaining links among content sources and providing for revision control. They may be used in conjunction with single sourcing, but some types of content management systems are not compatible with single sourcing.
Before composing this definition, we reviewed the extended definitions of content management systems offered by Rockley (2001), Rockley, Kostur, and Manning (2002), and Doyle (2007). Our goal was to provide respondents with a distilled description leading them to focus on a networked information system and not on a general information management process. (See Clark [2008] for a discussion of process versus technology in defining content management, as well as descriptions of general types of content management systems.)
We use the following terms and abbreviations to refer to the three possible situations that apply to technical communication work groups with regard to the use of single sourcing and content management systems:
- Single sourcing without a content management system (SS)
- Single sourcing with a content management system (SSwCM)
- No single sourcing but use of a content management system (CM)
Note that we use SS/CM as shorthand for “SS and/or CM”—in other words, whenever we refer to the group of respondents who reported using SS only, CM only, or SSwCM. In reporting our results, we often compare the group composed of all SS/CM respondents with the group composed of all whose work groups did not use any SS/CM method or tool. Within the main group of interest—the users of SS/CM—we often break down the results for the three subgroups: SS only, CM only, and SSwCM.
A few additional definitions are needed because it is impractical to discuss this topic without them. Extensible Markup Language (XML) is an open-source, application-independent markup language frequently used in (though not required by) tools across the spectrum of SS/CM applications and systems. XML is becoming a universal markup language for information development and exchange. Many times, people using XML-based tools are unaware of XML's role, as when one saves a document in Word 2007's default “.docx” format, which is a zip file containing XML components. Our survey included questions about the use of XML and its precursor, SGML (Standard Generalized Markup Language), as well as a question about three standards for implementing XML to develop and manage documentation: DocBook, Darwin Information Typing Architecture (DITA), and S100D (a standard used in the aerospace and defense industries).
Objectives and Methodology
Our study had the following four objectives, to:
- Produce a cross-sectional statistical profile of SS, CM, and SSwCM use by STC members
- Identify important factors perceived by STC members to be driving or impeding the adoption of SS, CM, and SSwCM methods and tools
- Gather data on the transition experiences of work groups after they adopted these methods and tools
- Learn whether and how these methods and tools are perceived by STC members using them to have impacts on efficiency, documentation usability, customer focus, and job stress
Development of the Survey
The survey was the central element of a multimodal research proposal that Dayton submitted to the STC Research Grants Committee, a group of prominent academics and practitioners with many years of experience conducting and evaluating applied research projects. Dayton revised the first formal draft of the survey in response to suggestions from the committee, which recommended to the STC Board that the revised proposal receive funding. The Board approved the funding in June 2007, and Dayton obtained approval for the study from the Institutional Review Board (IRB) for the Protection of Human Participants at Towson University in Maryland.
Based on several formal interviews and some informal conversations with technical communicators about single sourcing and content management methods and tools, Dayton revised the survey and solicited reviews of the new draft from three practitioners with expertise in the subject matter and from an academic with expertise in survey research. Dayton again revised the survey in response to those reviewers' suggestions. Hopper then converted the survey into an interactive Web-delivered questionnaire using Zoomerang (a copy of the survey that does not collect data may be explored freely at http://www.zoomerang.com/Survey/WEB22B38UWBJKZ).
Moving the survey from a page-based format to multi-screen Web forms proved challenging. Multiple branching points in the sequence of questions created five primary paths through the survey: no SS/CM, SS only, CM only, SSwCM, and academics. Respondents not using SS or CM were presented with 20 or 21 questions depending on whether their work group had considered switching to SS/CM methods and tools. Respondents in the three subgroups of SS/CM were presented with 30 to 33 questions, depending on their answers to certain ones. The version of the survey for academics contained 24 questions, but we ultimately decided to leave academics out of the sampling frame for reasons explained later.
For all paths through the survey, question types included choose one, choose all that apply, and open ended. All fixed choice questions included a final answer choice of “Other, please specify” followed by a space for typing an open-ended answer. The first complete draft of the Web-based survey was pilot tested by about 30 participants, which included practitioners, graduate students, and academics. The reported times for completing the survey ranged from less than 8 to 25 minutes. Testers who went through the path for academics and the path for those not using SS or CM reported the fastest completion times and offered the fewest suggestions. Testers answering the questions for those using SS/CM suggested some improvements in wording, formatting, and answer options, most of which we agreed with and made changes to address.
Deployment of the Survey
The version of the survey for academics was entirely different from the four variations for practitioners. Following the pilot test, we reassessed the pros and cons of fielding two surveys at the same time. We were particularly concerned that the number of academic respondents would be quite small unless we drew a separate sample of only academic members. After the STC Marketing Manager assured us that academics could be filtered from the membership database before drawing a sample, we decided to limit the sampling frame to practitioners. (The sampling frame is the total population of people from whom the random sample is drawn.)
The sampling frame consisted of about 13,500 STC members, about 3,000 fewer than the total membership at that time (May 2008). In addition to excluding academics, students, and retirees, the STC Marketing Manager also excluded STC members who had opted not to receive messages from third-party vendors. From the sampling frame of about 13,500 members, the STC Marketing Manager drew a random sample of 1,000 using an automated function for that purpose available in the STC office's membership database application.
Over 11 days, the Marketing Manager e-mailed to the sample four messages that we composed. The first e-mail went out on a Thursday: a brief message from STC President Linda Oestreich describing the survey and encouraging participation. The second e-mail was sent the following Tuesday, signed by us, inviting recipients to take the survey and providing a link to the consent form. (Researchers working for federally funded institutions are required by law to obtain the informed consent of anyone asked to participate in a research study.) Respondents accessed the survey by clicking the link at the bottom of the consent form. (Appendix C contains copies of the two e-mails mentioned above and the consent form.)
The Internet server housing the survey was configured to prohibit multiple submissions from the same computer. When a respondent completed the survey by clicking the Submit button on the final screen, a confirmation page displayed our thank-you message and offered respondents the option of e-mailing the STC Marketing Manager to be taken off the list of those receiving reminder e-mails. In addition, respondents could check an option to receive an e-mail from STC after the survey had closed, giving them an early look at the results.
We received data from 117 respondents within 24 hours of sending out the first e-mail with a link to the survey. Based on the response time data that we had obtained in previous online surveys, this level of initial response suggested that we were headed for a lower than anticipated response rate. Two days after our first e-mail with a link to the survey went out, the first reminder e-mail was sent—with a revised subject line and sender address. The initial two e-mails had been configured to have stc@stc.org as the sender, which we feared might be leading some recipients to delete it reflexively or to filter it to a folder where they would not see it until it was too late to take the survey. We arranged with STC staff to have the reminder e-mails show the sender as david_dayton@stc.org, an alias account. A second and final reminder was e-mailed the following Monday, 11 days after the advance notice e-mail went out.
The sequencing, timing, and wording of the four messages e-mailed to the 1,000 STC members in the sample were based on best practices for conducting Internet surveys (cf. especially Dillman, 2007). Because we did not have direct control over the sampling frame and the mass e-mails used to distribute the survey invitations, some aspects of the survey deployment did not meet best-practices standards; specifically, our e-mailed invitations lacked a personalized greeting and, for the first two e-mails, also contained impersonal sender-identifying information.
Response Rate
Two weeks after the first e-mail went out to STC members in the sample, the survey closed. We had received data from 276 practitioners who completed the survey. We will not report data from four other respondents who answered the version of the survey for academics, and we discarded partial data from 46 participants who abandoned the survey after starting to fill it out. Using the standard assumption that the 1,000 e-mailed survey invitations were all received by those in the sample, the response rate was 28%, slightly better than other recent STC surveys. (The last salary survey that STC invited 10,000 members to take in 2005 had a response rate of 23%. A sample survey conducted by a consulting firm hired in 2007 to collect members' opinions about STC publications had a response rate of 22%.) Our survey's response rate of 28% may represent a source of bias in the survey results. We comment on this briefly toward the end of the summary of results and discuss it in some depth in Appendix B, where we review recent research and thinking about low response rates from the social science literature.
Data Analysis Methods
Data from submitted surveys were collected in a text file on the Zoomerang.com server and downloaded after the survey closed. Microsoft Excel 2007 was used to create frequency tables and bar graphs to examine descriptive statistics for each survey question. Data from key variables were sorted by technology type—SS only, CM only, or SSwCM—and tested for significant differences or associations using statistical software to run the most appropriate procedures based on the level of the data. Standard measures were used to calculate the strength of any statistically significant differences or associations (p = .05). Please note that data are rounded to whole numbers using the “round up for odd, down for even” rule when the exact proportion produces a 5 after the decimal point; thus, the whole numbers for the same item will occasionally not add up to 100%. For example, 18.5% will be reported as 18%, while 19.5% will be reported as 20%. This is a standard rounding protocol intended to produce greater clarity in reporting the results for this type of survey.
Summary of Results
In this section, we present a summary of the survey data organized under headings that highlight the most noteworthy results. Readers wishing to explore the survey data in more depth may visit the STC Live Learning Center (www.softconference.com/stc/), which has an audio recording and PowerPoint slidedeck of our presentation at the STC 2009 Summit.
Four of Five Were Regular Employees; Half Worked in High-tech
The group profile of our 276 respondents in terms of employment status and industry seems typical of the STC membership before the current economic recession: 81% were regular employees; 18% were contractors, consultants, freelancers, or business owners; and 2% were unemployed. Respondents worked for a wide range of industries, though a little more than half worked in industries commonly clustered under the rubric “high-technology”: companies making or providing software, computer and networking hardware, software and IT services, and telecommunications products and services.
Slightly More Than Half Worked in Large Companies
We asked respondents to categorize the size of the company they worked at. Table 1 shows that the range of company sizes was weighted slightly (55%) toward companies with more than 500 employees, and the largest category proportionately is 10,000 or more employees, with 25%. (The Small Business Administration most often uses 500 employees as the maximum size company allowed to access its programs.) Table 1 includes Census Bureau data for the entire U.S. economy in 2004 as comparative data.
Table 1. Company Size Reported by Respondents Compared with 2004 U.S. Census Data
|
Company Size |
% of 276 STC respondents |
% of U.S. Census Data 2004* |
|---|---|---|
|
1 to 4 |
7% |
5% |
|
5 to 9 |
2% |
6% |
|
10 to 19 |
2% |
7% |
|
20 to 99 |
14% |
18% |
|
100 to 499 |
21% |
15% |
|
500 to 999 |
7% |
5% |
|
1,000 to 9,999 |
22% |
18% |
|
10,000 or more |
25% |
26% |
| Source: Statistics about Business Size (including Small Business) from the U.S. Census Bureau, Table 2a. Employment Size of Employer and Nonemployer Firms, 2004. Accessed August 16, 2009, at http://www.census.gov/epcd/www/smallbus.html | ||
Half Used SS Only, CM Only, or SS With CM—and Half Used No SS/CM
Of the 276 respondents, 139 (50%) reported that they did not use SS/CM methods and tools, and 137 (50%) reported that they did (see Figure 1). In the SS/CM group, SSwCM users were the most numerous (55, or 20% of all respondents), followed by SS only (47, 17%) and CM only (35, 13%).
Figure 1. Use of SS/CM by 276 Survey Respondents
As Figure 2 shows, about two-thirds of SS/CM users reported that their work groups produced more than half their output using SS/CM methods and tools. One in five, however, reported that their work group used SS/CM to produce 25% or less of their output, a finding consistent with the data collected on recentness of SS/CM adoption and the average time reported for reaching certain benchmarks for proportion of total output using SS/CM methods and tools. (Those results are reported in subsequent tables and figures.)
Figure 2. Proportion of Total Information Product Output Using SS/CM
About 1 in 4 Used XML and/or SGML; About 1 in 9 Used DITA
All 276 respondents answered a question asking them to identify the types of information products their work groups produced. Seventy-six (28%) checked the answer “content developed using XML or SGML.” Respondents using SS/CM (n = 137) were presented with another question asking them to indicate if their work group used XML and/or SGML. Figure 3 graphs the results from that question, showing that about half the SS/CM respondents produced content using XML and/or SGML. Three out of four in that group of SS/CM users indicated their work group's system used XML alone, while most of the others indicated a system using both XML and SGML.
Figure 3. Use of XML and SGML by 137 SS/CM Respondents
Another question presented to SS/CM respondents asked them to indicate which, if any, documentation standard their work group used. About 2 of 3 SS/CM respondents (64%) reported that their work group used no standard. About 1 in 5 (21%) indicated that they used DITA, and one person used both DITA and DocBook. The 30 DITA-using respondents, then, were 11% of all survey respondents, or 1 in 9.
About 1 in 10 Reported a Failed SS/CM Implementation
Twenty-four respondents (9% of N = 276) reported that they had been part of a work group whose attempt to implement an SS/CM system had failed. Seven indicated that a CM system was involved, and six wrote that it was the wrong tool for their work group, citing one or more reasons. Three respondents indicated that an SS tool had failed, two saying that the SS tool had not performed to expectations and the third saying that lack of management support led to failure of the project. Fourteen respondents did not specify which type of tool was involved in the failed project, and for this subgroup no single reason for the failure predominated. Poor fit, difficulty, and cost were the main reasons cited for the failed implementations.
Almost Half the SS/CM Work Groups Had Used Their System for Two Years or Less
The survey asked those using SS/CM how long ago their work group had started using their current SS/CM system. Figure 4 shows that 45% of the SS/CM users' work groups had been using their SS/CM system for less than two years, and 24% had been using their system for less than a year. When asked how long the work group had researched options before deciding on its SS/CM system, 103 respondents provided an estimate in months. Setting aside an outlier (40 months), the range of answers was 0 to 24 months, with a median of 4, a mean of 6.04, and a standard deviation of 6.03 (see Table 2).
Figure 4. How Long Ago Did Work Group Begin Using SS/CM System?
The survey also asked SS/CM users to estimate how long (in months) it took their work group to reach the point of producing 25% of their information products using their SS/CM system. Estimates (n = 97 valid) ranged from 0 to 28 months, with a median of 4 months, a mean of 6.4, and a standard deviation of 6.25. Of the 137 respondents using SS/CM, 55% reported that their work group had completed their SS/CM implementation; 45% reported that their group was still working to complete their SS/CM implementation (however they defined that milestone, which is not usually defined as 100% of information production output, as shown in Figure 2). Table 2 reveals that the average time it takes a work group to implement an SS/CM system seems reasonable: most work groups adopting SS/CM systems complete their implementation in well under a year. However, some work groups experience very long implementation times.
Table 2. Estimated Months to Research Options, to Reach 25% Production with SS/CM, and to Complete the Implementation Process
|
Measures of central tendency |
Months during which work group researched SS/CM options |
Months before work group produced 25% of its output with SS/CM |
Months it took to complete SS/CM implementation (historical) |
Months it will take to complete implementation (projection) |
|---|---|---|---|---|
|
Median |
4 |
4 |
6 |
10.5 |
|
Mean |
6.1 |
6.4 |
7.9 |
10.7 |
|
SD |
5.96 |
6.25 |
7.07 |
7.95 |
|
Range |
0 to 24 |
0 to 28 |
0 to 28 |
0 to 24 |
Caution must be exercised in comparing estimates by those working toward completion of SS/CM implementation with the historical estimates by those looking back at that completed milestone. For those in the “not done” group, we do not know how long SS/CM projects had been underway when they estimated how long it would be before their work group completed its implementation. With that caveat in mind, we observe that the data in Table 2 are consistent with what we know about human nature: those looking ahead to completion of SS/CM implementation tended to see the process taking somewhat longer than those looking back in time.
SS/CM Respondents Reported Many Activities to Prepare for Transition
The survey asked SS/CM users what activities their work group engaged in to help them make the transition to SS/CM, and 83% in the SS/CM group provided answers. Figure 5 shows that SS/CM work groups engaged in a wide range of research and professional development activities to pave the way for adoption and implementation of SS/CM systems. As we would expect, about half of the work sites gathered information from vendor Web sites. The next most mentioned activity was trying out the product, which 37% said their work group did. Only slightly fewer (31%) indicated that members of their work group attended conferences and workshops to learn more about SS/CM systems. About 1 in 4 (23%) indicated that their work group hired a consultant to help them make the transition.
Figure 5. Transition to SS/CM Activities Reported by SS/CM Respondents
Top Drivers: Faster Development, Lower Costs, Regulatory and Compliance Pressures, Translation Needs
On one question, the 137 SS/CM users indicated which listed business goals influenced the decision to adopt the SS/CM system their work group used. The next question asked them to select the business goal that was the most important driver of the decision to adopt the SS/CM system. Figure 6 charts the results from these related questions. On the “choose all that apply” question, the business goal most often selected was providing standardization and consistency (73%). Three other business goals were indicated as influential by more than half of the SS/CM group: speeding up development (57%), lowering costs (56%), and providing more usable and useful information products (52%).
Figure 6. Business Goals Driving Decision to Implement SS/CM System
In identifying the single most important business goal driving the decision to adopt the SS/CM system, about 1 in 5 respondents picked one of the first three factors listed above, with lowering costs edging out standardization and development speed as the most-picked factor. About 1 in 8 picked either lowering translation costs specifically or providing more usable and useful information products as the most important factor; only 6% chose responding to regulatory or compliance pressures as the single most important driver of adoption.
SSwCM Respondents Reported Significantly Larger Work Groups
Table 3 shows that respondent work group sizes were similar for three groups: No SS or CM use; use of SS only; and use of CM only. However, the work group size reported by SSwCM users was significantly different.
Table 3. SSwCM Users Reported Significantly Larger Work Group Sizes*
|
Measures of central tendency |
No SS/CM |
SS only |
CM only |
SS with CM |
|---|---|---|---|---|
|
Median |
4.00 |
5.00 |
5.00 |
12.00 |
|
Mean |
6.91 |
8.24 |
9.45 |
18.00* |
|
SD |
9.919 |
11.478 |
11.869 |
17.747 |
|
Range |
1 to 75 |
1 to 70 |
1 to 50 |
1 to 65 |
| The null hypothesis that differences in work group size are due to chance was rejected: A one-way Welch's variance-weighted ANOVA was used to test for differences among the group sizes reported by respondents in the four categories, and these were found to differ significantly F (3, 86.8) = 6.19, p = .001. Tamhane post hoc comparisons of the four groups show that work sizes reported by those in the SSwCM category (M = 18.0) differ significantly from those of the No SS or CM category (M = 9.91, p = .000); those of the SS only category (M = 8.24, p = .009); and those of the CM only category (M = 9.45, p = .053). | ||||
SS/CM and Non-use Groups Varied Significantly by Company Size
Knowing that larger work group sizes predict a significantly greater likelihood of using SS/CM methods and tools, we would expect the same to hold true, generally, for the association between company size and likelihood of using SS/CM. That is the case, though the association is not as strong as work group size. Chi square analysis revealed that the proportions shown in Table 4 are significantly different, ?2 (9, N = 275) = 25.283, p = .003. Somers' d, used to test the strength of significant chi square associations for ordinal by ordinal data, had a value of .17, which is noteworthy, though weak. (In other words, knowing the size of a respondent's company reduces prediction errors about which SS/CM subgroup the respondent is in by 17%.)
Table 4. SS/CM Use Categories Cross-Tabulated with Company Size Categories
|
Category of SS/CM Use |
1–99 |
100–999 |
1,000–9,999 |
10,000 or more |
Totals |
|
|---|---|---|---|---|---|---|
|
No SS/CM |
Count |
42 |
42 |
25 |
30 |
139 |
|
% within category |
30% |
30% |
18% |
22% |
100% |
|
|
SS only |
Count |
13 |
18 |
9 |
7 |
47 |
|
% within category |
28% |
38% |
19% |
15% |
100% |
|
|
CM only |
Count |
8 |
7 |
7 |
13 |
35 |
|
% within category |
23% |
20% |
20% |
37% |
100% |
|
|
SSwCM |
Count |
5 |
10 |
20 |
19 |
54 |
|
% within category |
9% |
19% |
37% |
35% |
100% |
|
|
Total |
Count |
68 |
77 |
61 |
69 |
275 |
|
% within category |
25% |
28% |
22% |
25% |
100% |
|
| * Null hypothesis that differences in proportions across columns are due to chance was rejected: ?2 (9, N = 275) = 25.283, p = .003; Somers' d = .172 | ||||||
SS/CM Was Significantly Associated with Greater Translation Needs
A question presented to all respondents asked, “Regarding your work group's information products: Into how many languages are some or all of those products translated?” Table 5 sorts the answers into the four categories formed by the fixed choices, which ranged from 0 languages to 10 or more languages. Chi square analysis revealed that the proportions shown in Table 5 are significantly different, ?2 (9, N = 276) = 34.563, p = .000. Goodman and Kruskal's tau, a proportional reduction in error directional measure of association for nominal by nominal data, was 0.51 with the SS/CM category as the dependent variable. (Knowing the number of languages for translation reduces errors in predicting the SS/CM category by half.) These results strongly support the perception among many technical communicators that translation needs are often a critically important factor in justifying the costs of moving to SS and/or SSwCM systems.
Table 5. SS/CM Category Cross Tabulated with Number of Languages Translated*
SS/CM Groups Differed Significantly on Some Likert-type Items About Impacts
The survey presented the SS/CM users with a series of 10 Likert-type items about perceived impacts of using SS/CM. These 137 respondents picked an answer on a five-point scale ranging from strongly disagree (value of 1) to strongly agree (value of 5). The mean ratings elicited by the 10 statements are shown in Figure 7. Pairwise comparison using the Kruskal-Wallis non-parametric test of independent groups showed significant differences between groups, which are footnoted in Figure 7. These statistically significant differences can be summed up as follows:
- Respondents whose work groups used single sourcing without content management (SS) agreed more strongly that their system “has helped speed up development of information products” than respondents from the other two groups—content management without single sourcing (CM) and single sourcing with content management (SSwCM).
- CM respondents agreed less strongly than respondents from the other two groups that their system “has helped speed up development of information products.”
- SS respondents more strongly agreed than SSwCM respondents that their system “has made our routine work less stressful overall.”
- SSwCM respondents more strongly agreed than respondents using SS only or CM only that their system “has improved the usability of our information products.”
Figure 7. Mean Rating of SS/CM Users on 10 Likert-Type Statements About SS/CM Impacts
Half the SS/CM Users Reported Significant Downsides or Tradeoffs
The survey asked those using SS/CM systems, “Has your work group and/or company experienced any significant downsides or tradeoffs resulting from switching to its SS/CM system?” Seventy-two of the 137 respondents (53%) answered “Yes” and also typed comments into a text-entry space. We did an initial coding of the comments and then further reduced the categories with a second round of coding, which produced the results shown in Table 6. Table 7 contains a representative sample of the comments in each of the top six categories.
Table 6. Comments on Downsides of SS/CM Implementation: Count by Category
|
Category into which comment was sorted |
n = 72 |
% of 137 |
|---|---|---|
|
Awkward production/slower production/more work for writers |
23 |
17 |
|
Difficult or slow transition/learning curve/team member resistance |
22 |
16 |
|
Bugs and technical glitches |
13 |
10 |
|
Lack of ability to customize |
5 |
4 |
|
Expense |
3 |
2 |
|
Garbage in, garbage out |
3 |
2 |
|
Technical skills demands; loss of process control; too early to tell |
1 each |
1 |
Table 7. Sample Comments on Downsides of SS/CM Implementation, for Top Six Categories from Table 6
|
Awkward production / slower production / more work for writers |
|
|
Difficult or slow transition / learning curve / team member resistance |
|
|
Bugs and technical glitches |
|
|
Lack of ability to customize |
|
|
Expense |
|
|
Garbage in, garbage out |
|
One in Four SS/CM Users Said That Their Work Group was Considering a Change in Tools
Thirty-nine or 28% of the SS/CM users indicated that their work group was considering a change to a different SS/CM system. In an open-ended follow-up question, 26 respondents mentioned specific tools under consideration. DITA was mentioned in nine responses; other tools mentioned more than one time were Structured FrameMaker (5 times), MadCap Flare (4), SharePoint (3), RoboHelp (2), and XMetal (2).
Half of the No-SS/CM Work Groups Had Considered SS/CM, but Few Planned to Adopt
In addition, about 1 in 3 reported that their work group had never considered switching to SS/CM, and about 1 in 10 were not sure or gave ambiguous explanations after checking “Other.” For 66 respondents (47%) in the no-SS/CM group who answered a follow-up question about factors driving their work group to consider using SS/CM, the most important factors were speeding up development (71% of n = 66), providing standardization and consistency (68%), and cutting costs (61%). These results are similar to those from SS/CM respondents (see Figure 6).
The 66 non-SS/CM respondents reporting that their work groups had considered SS/CM were asked to explain what their group had concluded about the feasibility of adopting SS/CM. About half these respondents mentioned as obstacles the money, time, and/or resources required to move forward with a transition to SS/CM. About 1 in 5 indicated that their work group or management concluded that SS/CM was not practical for them or not needed. Another 1 in 5 indicated that no decision had yet been made about the feasibility of switching to SS/CM.
Respondents Reported Producing a Diverse Array of Information Products
All respondents were presented with a long list of information products and checked all the ones their work group produced (see Table 8). Not surprisingly, PDF files and technical content documents were the top categories, selected by 9 out of 10 respondents. About 3 out of 4 said their work groups produced Microsoft Word files and/or content with screen shots. Two other types of products were selected by over half the respondents: HTML-based help content and instructional content. Far fewer respondents indicated their work groups produced multimedia and/or interactive content, such as videos, animation, simulations, and interactive Flash or Shockwave content.
We intended for the product categories to overlap and to represent as broad a spectrum of information products as possible, but respondents could add other types in an open-ended “other” follow up question. We examined the 29 responses to the “other” question and identified 12 responses representing types of information products not already checked by the respondent, such as reports, proposals, forms, posters, and so forth.
Table 8. Types of Information Products Reported by All Respondents
|
% |
Information Products |
N = 276 |
|---|---|---|
|
91 |
PDF files |
252 |
|
91 |
Technical content documents |
252 |
|
79 |
Content with screen shots |
217 |
|
72 |
Microsoft Word files |
200 |
|
57 |
Help content (HTML Help, Web Help, etc.) |
157 |
|
56 |
Instructional content |
156 |
|
46 |
Content with technical diagrams or illustrations |
126 |
|
42 |
Other Web page-delivered content |
117 |
|
28 |
XML or SGML content |
76 |
|
27 |
e-learning content |
75 |
|
24 |
Knowledgebase topics |
65 |
|
24 |
Video or animation content |
65 |
|
18 |
Software demonstrations or simulations |
50 |
|
17 |
Flash Player interactive content |
47 |
|
14 |
Content with 3D models |
39 |
|
5 |
Content for mobile devices |
15 |
|
5 |
Shockwave Player interactive content |
15 |
|
4 |
Miscellaneous other not counted in above |
12 |
Microsoft Word Was the Most-used Primary Authoring Tool
All 276 respondents answered this question by typing into a text-entry box: “What is your work group's primary tool for textual content authoring/editing?” Naturally, we had to categorize the answers, shown in Table 9. About 1 in 2 respondents (46%) identified Microsoft Word as their work group's primary authoring/editing tool. Approximately 1 in 3 (30%) named Adobe FrameMaker. The remaining quarter of the respondents listed a variety of tools, including Arbortext Editor (4%), RoboHelp (3%), Author-it (3%), XMetal (2%), and InDesign (2%).
Table 9. Primary Authoring/Editing Tool
|
% |
Tool |
N = 276 |
|---|---|---|
|
46 |
Microsoft Word |
127 |
|
30 |
Adobe FrameMaker |
83 |
|
4 |
Arbortext Editor (Epic Editor) |
12 |
|
3 |
Adobe RoboHelp |
9 |
|
3 |
Author-it |
8 |
|
2 |
XMetaL |
7 |
|
2 |
Adobe InDesign |
5 |
|
1 |
XML |
3 |
|
1 |
MadCap Flare |
2 |
|
7 |
Misc. other |
19 |
Conclusions
The survey results summarized above provide a snapshot—taken in May 2008—depicting STC members' use of single sourcing and content management methods and tools. These results are the first publicly available data from a random sample survey on this topic. In this section, we discuss the most important conclusions to be drawn from the data.
Has Single Sourcing and/or Content Management Reached a Critical Mass?
Everett M. Rogers (1995) depicted the rate of adoption for any given innovation as a normal, bell-shaped curve and designated categories of adopters—from early adopters to laggards—based on their postulated time-to-adopt relative to the average time for all potential adopters (see Figure 8). Rogers further postulated that a “critical mass” had to adopt an innovation before it could “take off”—reaching what popular author Malcolm Gladwell (2000) famously termed “the tipping point.”
Figure 8. Rogers's Innovation Adopter Categories Depicted as Normal Distribution of Time-to-Adoption (Rogers, 1995, p. 262)
If all varieties of SS/CM are considered together as the innovation, the answer about critical mass is a confident yes: half of our respondents reported using SS, CM, or SSwCM. In addition, as shown by the data on how long groups had been using their SS/CM system (see Figure 4), the pace of adoption of all three categories of SS/CM had picked up somewhat during the 2 years prior to the survey—from about mid-2006 to mid-2008. The current recession began in December 2007 (National Bureau of Economic Research, 2008). Undoubtedly, the recession has put a damper on the spread of SS/CM among technical communication work groups over the past 2 years. We think it is likely, however, that the recession may have had less impact on the adoption of SS systems, which generally have a lower price tag, than on the more expensive SSwCM systems.
If we regard each set of SS/CM methods and tools as a distinct innovation competing with the others, then our answer about critical mass, based on the Figure 1 data, becomes maybe for SS only and for SSwCM: Those methods and tools appear to have reached a critical mass of adopters. However, the results suggest that CM without single sourcing did not seem destined for widespread adoption in technical communication. In sum, our survey shows that as of mid-2008 STC members had moved into the Early Majority phase (Figure 8 ) for SS only and SSwCM, but CM by itself was still in the Early Adopter phase. Likewise, with regard to XML adoption, STC members were in the Early Majority phase, but for DITA they were in the Early Adopter phase (see Figure 3 and related explanatory text).
Are Larger Companies More Likely to Use SS/CM?
Yes—see Table 4—but the strength of the statistically significant association is weaker than some would predict. We found a stronger association between work-group size and likelihood of using SS/CM. And, of course, we come back to the problem of conflating all types of SS/CM methods and tools: The cost of adoption in time and money will vary widely depending on the specific solution adopted, adapted, and/or developed. Some SSwCM systems are expensive, and only companies with deep pockets can afford them. On the other hand, a small work group with one or two technically savvy and resourceful members could develop an SS-only or even an SSwCM system with relatively low-cost and/or open-source tools.
Are Translation Requirements a Big Driver of SS/CM Adoption?
Absolutely, yes: See Table 5. Our data support what anyone would have assumed who has followed this topic at STC conferences. However, translation is not the top driver of SS/CM adoption, as demonstrated in Figure 6, which shows that three business goals were picked about evenly as the most important driver of the decision to adopt an SS/CM system: Lowering costs generally, speeding up development, and providing standardization or improving consistency.
What Are the Biggest Surprises in the Survey Results?
For us, the biggest surprise was that only 1 in 10 respondents reported that they had been involved in a work group whose attempt to implement an SS/CM system had failed. On more than one occasion, one of us (Dayton) has heard prominent consultants at STC conferences estimate failure rates for SS/CM projects at 50% and higher. We think the data from our survey probably underestimates the actual failure rate for such projects, but we also suspect that these results mean that failure rates are commonly overestimated. This may be explained by different notions of what constitutes a failed project. Half of our survey's respondents who reported no SS/CM use also reported that their work group had considered a switch to SS/CM but had no plans to move in that direction. This suggests that many work groups investigate SS/CM options, including contacting consultants, but end up deciding to stay with the methods and tools they have, often without trying to implement an SS/CM system. To a consultant, that may count as a failure to implement, but to insiders it may simply be a successful conclusion to a deliberative process focused on feasibility.
Another surprise was that 1 in 4 respondents in work groups using SS/CM was considering a change in methods and tools and that 1 in 2 reported significant downsides to their current SS/CM methods and tools. We did not expect that high a level of dissatisfaction with SS/CM methods and tools; on the other hand, we did not ask non-users of SS/CM a similar question about perceived downsides of their methods and tools.
What Else in the Results Deserves to Be Highlighted?
Microsoft Word and FrameMaker were by far the most-used primary authoring tools of the survey respondents, and more than three times as many respondents produced PDF files as produced content using XML or SGML.
We also think that the data on the Likert-type agreement-disagreement items are intriguing: SS-only respondents were significantly more in agreement that their system had speeded up their work while reducing their work-related stress. SSwCM respondents, however, were significantly more in agreement that their system had made work groups more focused on information usability issues. These results tempt us to speculate that the added complexity of implementing single sourcing through a content management system adversely impacts perceptions of overall efficiency and stressfulness while bolstering perceptions that the work group is giving more attention to the usability of its information products. Perhaps implementing SSwCM is more likely to compel work groups to re-invent their information development processes, leading to more user-centered analysis and testing of their information products.
Is It Likely That This Survey Underestimates Use of SS/CM by STC Members?
For surveys of the general public, textbooks about social science research instruct that a low response rate, commonly specified as below 50% (Babbie, 2007, p. 262), warrants caution in assuming that data from the survey accurately represent the results that would be produced if data could be gathered from all members of the represented group. Our survey's response rate of 28% must be viewed as a limitation of the study: Because we lack information about the nonrespondents to the survey, we cannot know whether they, as a group, differ significantly from respondents in regard to the topics covered by the survey. The discussion about how likely it is that the survey's results accurately represent the experiences and attitudes of STC members in 2008 must be grounded in logical imputation.
We do not think the results underestimate STC members' use of single sourcing and content management in the spring of 2008. Indeed, we think that it seems just as likely that the survey overestimates SS and CM use by STC members. We make that argument in Appendix B, for those who may be interested in a review and discussion of research supporting the proposition that low survey response rates do not automatically mean questionable data quality. Our examination of the literature on that topic has bolstered our confidence that our survey presents a reasonably accurate snapshot of STC members' experiences and opinions related to single sourcing and content management.
From the Survey Results, What Dare We Predict About the Future of SS/CM?
The survey results make for a rather cloudy crystal ball. Nevertheless, adding them to what we know from half a decade of following the information about SS/CM disseminated in the publications and at the conferences of technical communication practitioners and academics, we feel confident in making these general predictions:
- Single sourcing will slowly but steadily gain wider acceptance among technical communication workgroups. Single sourcing seems destined to reach a significantly larger proportion of adopters than single sourcing with content management—barring a technological breakthrough that makes SSwCM systems significantly cheaper and easier to install, use, and maintain. Perhaps, though, one or more popular SS tools such as Adobe FrameMaker and MadCap Flare will evolve into true SSwCM solutions, altering the SS/CM marketplace quite dramatically.
- Pushing XML-enabled single sourcing to the tipping point may take the arrival, or the more effective marketing, of user-friendly and affordable plug-in tools for Microsoft Word, which was by far the most-used authoring tool of STC members in May 2008.
- The number of eventual SS/CM adopters in technical communication may be somewhat lower than SS/CM vendors and consultants anticipate. Already, Web 2.0 and social media/networking methods and tools are stealing the spotlight from SS/CM topics at the leading conferences attended by technical communicators.
That last conjecture seems a suitably provocative note to end on. Standardized structure and control are at the heart of the SS/CM paradigm, but those qualities are anathema to the Web 2.0/social networking paradigm. What's going on here? Could it be that many companies find today that they need technical communicators to produce a continuous stream of just-in-time, variously structured, often transient, multimedia content—as much or more than they need them to produce highly regulated and uniform topics in a database whose information, as well as its meta-information, is composed almost entirely of words?
This question, in simpler forms, will become the focus of much discussion among technical communicators. It represents only one of several obvious directions for further research related to the incessant search for better, cheaper, and faster ways of creating useful and usable technical information products.
References
Ament, K. (2003). Single sourcing: Building modular documentation. Norwich, NY: William Andrew Publishing.
Babbie, E. R. (2007). The practice of social research, 11th ed. Belmont, CA: Thomson Wadsworth.
Clark, D. (2008). Content management and the separation of presentation and content. Technical Communication Quarterly, 17, 35–60.
Dillman, D. A. (2007). Mail and Internet surveys: The tailored design method (2nd ed.). Hoboken, NJ: Wiley.
Doyle, B. (2007). Selecting a content management system. Intercom, 54(3): 9–13.
Gladwell, M. (2000). The tipping point: How little things can make a big difference. Boston: Little, Brown.
Hall, W. P. (2001). Maintenance procedures for a class of warships: Structured authoring and content management. Technical Communication, 48, 235–247.
Happonen, T., & Purho, V. (2003). A single sourcing case study. Presentation (slides) at STC 50th annual conference (Dallas, TX, May 18–21). Retrieved from https://www.stc.org/edu/50thConf/dataShow.asp?ID=110
McCarthy, J. E., & Hart-Davidson, W. (2009). Finding usability in workplace culture. Intercom, 56(6), 10–12.
National Bureau of Economic Research. (2008, December 11). Determination of the December 2007 peak in economic activity. Retrieved from http://wwwdev.nber.org/dec2008.pdf.
Petrie, G. (2007). Industrial-strength single-sourcing: Using topics to slay the monster project. Presentation (slides) at 54th annual conference of the Society for Technical Communication (Minneapolis, MN, May 13–16). Retrieved from https://www.stc.org/edu/54thConf/dataShow.asp?ID=27.
Pettit Jones, C., Mitchko, J., & Overcash, M. (2004). Case study: Implementing a content management system. In G. Hayhoe (ed.), Proceedings of the 51st annual conference of the Society for Technical Communication (Baltimore, Maryland, May 9–12). Arlington, VA: STC. Retrieved from https://www.stc.org/ConfProceed/2004/PDFs/0048.pdf.
Pierce, K., & Martin, E. (2004). Content management from the trenches. In G. Hayhoe (ed.), Proceedings of the 51st annual conference of the Society for Technical Communication (Baltimore, MD, May 9–12). Arlington, VA: STC. Retrieved from https://www.stc.org/ConfProceed/2004/PDFs/0049.pdf.
Rockley, A. (2001). Content management for single sourcing. In Proceedings of the 48th annual conference of the Society for Technical Communication (Chicago, IL, May 13–16). Arlington, VA: STC. Retrieved from https://www.stc.org/ConfProceed/2001/PDFs/STC48-000171.pdf.
Rockley, A., Kostur, P., & Manning, S. (2002). Managing enterprise content: A unified content strategy. Indianapolis, IN: New Riders.
Rogers, E. M. (1995). Diffusion of innovations, 4th ed. New York: Free Press.
Welch, E. B., & Beard, I. (2002). Single sourcing: Our first year. In G. Hayhoe (ed.), Proceedings of the 49th annual conference of the Society for Technical Communication (Nashville, Tennessee, May 5–8). Arlington, VA: STC. Retrieved from https://www.stc.org/ConfProceed/2002/PDFs/STC49-00070.pdf.
About the Authors
David Dayton is an Associate Fellow of STC. He has worked in technical communication since 1989 as a technical writer and editor, Web content designer and usability professional, and university teacher and researcher. He conducted this research while he was a faculty member of the English Department at Towson University. He recently left academe to join the International Affairs and Trade team of the U.S. Government Accountability Office, where he works as a Communications Analyst. E-mail address: dr.david.dayton@gmail.com
Keith B. Hopper has taught in the master's program in Information Design and Communication at Southern Polytechnic State University since 2001. An associate professor there, he also teaches in the Technical Communication undergraduate program. Recently, he launched an innovative master's degree program in Information and Instructional Design: http://iid.spsu.edu. He holds a PhD in Instructional Technology from Georgia State University. E-mail address: khopper@spsu.edu
Dayton manuscript received 26 February 2010, revised 28 August 2010, accepted 8 September 2010.
Appendix A: An Annotated Bibliography
Because our survey was about methods and tools that have been much discussed in conferences and the literature of the field for over a decade, we did not begin our report with an introductory literature review—the conventional way of justifying a new study and showing its relation to prior research and theory. Instead, we provide this brief annotated bibliography. We selected these sources as recent and useful starting points for delving into the abundant literature by technical communicators discussing single sourcing and content management.
Dayton, D. (2006). A hybrid analytical framework to guide studies of innovative IT adoption by work groups. Technical Communication Quarterly, 15, 355–382.
This article reports a case study of a medium-sized company that carried out a user-centered design process, complete with empirical audience research and usability tests, to put all its technical reference, troubleshooting, training, and user assistance information into a single-source, database-driven content management system. The case study is interpreted through the lens of a hybrid analytical framework that combines and aligns three distinct theoretical traditions that have been used to guide technology adoption and diffusion studies.
Dayton, D. (2007). Prospectus for a multimodal study of single sourcing and content management. In IPCC 2007: Engineering the future of human communication. Proceedings of the 2007 IEEE International Professional Communication Conference (IPCC) held in Seattle, Washington, Oct. 1–3, 2007. Piscataway, NJ: IEEE.
This proceedings paper describes the research project funded by STC in 2007, of which the survey reported in our article is the major part. It contains a justification for the focus of the study based in a traditional review of the literature.
Kastman Breuch, L. (2008). A work in process: A study of single-source documentation and document review processes of cardiac devices. Technical Communication, 55, 343–356.
This article from the STC journal documents a case study with details on implementation and impacts that offer a healthy practical counterpoint to the more abstract and theoretical perspectives that dominate the chapters in the Pullman and Gu collection. Kastman Breuch is particularly interested to explore the impacts of single sourcing (implemented through a content management system) on the document review process: “Both of these practices influence the roles and identities of technical writers as individual authors. What happens when we examine the impact of both practices—document review processes and single sourcing—together?” (p. 345).
Pullman, G., & Gu, B. (Eds.). (2008). Content management: bridging the gap between theory and practice. Amityville, NY: Baywood Pub. Co.
A collection of 11 articles originally published in a special issue of Technical Communication Quarterly, this book will appeal primarily to those seeking an in-depth, critical exploration of content management systems. The book's editors define CM broadly, and none of the chapters specifically focus on single sourcing. An online copy of the book's introduction is available at the publisher's Web site: http://www.baywood.com/intro/378-9.pdf.
Rockley, A. (2001). The impact of single sourcing and technology. Technical Communication, 48, 189–193.
This article in the STC's journal was the first to propose a comprehensive scheme for defining types of single sourcing. Rockley described four distinct levels of single sourcing, with level 2 corresponding to what we have defined as single sourcing without content management. Level 3 corresponds to what we have defined as content management: “Information is drawn from a database, not from static, pre-built files of information” (p. 191). Rockley equates level 4 with advanced electronic performance support systems that are not practical to implement in most user-assistance scenarios.
Williams, J. D. (2003). The implications of single sourcing for technical communicators. Technical Communication, 50, 321–327.
This article by a practicing technical communicator provides an excellent starting point for readers new to the topic of single sourcing. Williams provides concise but comprehensive summaries of key articles and books from 2000 to 2003 and provides a well-selected further reading list that includes articles from 1995 to 2002.
Appendix B: New Thinking About Survey Response Rates
Researchers have recently called into question whether a survey response rate of 60% to 70% should be considered, by itself, to ensure that the results are more trustworthy than those from a survey with a much lower response rate (Curtin, Presser, & Singer, 2000; Keeter et al., 2000; Merkle & Edelman, 2002). Groves, Presser, and Dipko (2004) sum up the challenge to the conventional wisdom on response rates: “While a low survey response rate may indicate that the risk of nonresponse error is high, we know little about when nonresponse causes such error and when nonresponse is ignorable” (p. 2).
“Emerging research,” Radwin wrote (2009), “shows that despite all the hand-wringing about survey nonresponse, the actual effect of response rate on survey accuracy is generally small and inconsistent, and in any case it is less consequential than many other serious but often ignored sources of bias” (para. 4). Radwin cites a study by Visser, Krosnick, Marquette, and Curtin (1996) that compared the pre-election results of mail surveys conducted from 1980 through 1994 with the results of parallel telephone surveys conducted in the same years. The average response rate of the mail surveys was 25% while the telephone surveys reported estimated response rates of 60% to 70%. Based on response rate alone, conventional wisdom would predict that the telephone surveys were significantly more accurate than the mail surveys, but the opposite was the case. The mail surveys consistently outperformed the telephone surveys on accuracy. Visser et al. concluded that “to view a high response rate as a necessary condition for accuracy is not necessarily sensible, nor is the notion that a low response rate necessarily means low accuracy” (p. 216).
We believe that what Visser et al. (1996) found to be true of surveys of the electorate is even more likely to hold true for surveys such as ours whose sampling frame is confined to the members of a professional organization. Almost four decades ago, Leslie (1972) noted that “when surveys are made of homogeneous populations (persons having some strong group identity) concerning their attitudes, opinions, perspectives, etc., toward issues concerning the group, significant response-rate bias is probably unlikely” (p. 323). In their recent meta-analysis of studies on nonresponse error in surveys, Groves and Peytcheva (2008) concluded that “the impression that membership surveys tend to suffer from unusually large nonresponse biases may be fallacious” (p. 179), even though relatively low response rates for such surveys have become a well-known problem.
Rogelberg et al. (2003) stress the self-evident point, often forgotten in discussions on this topic, that survey nonresponse is not the same as survey noncompliance—the purposeful refusal to take a survey. If a sizable number of our e-mailed survey invitations never reached the intended recipients, because of spam blockers, for example, or filters created by recipients to delete e-mails from certain senders, then the actual response rate would be higher—though by how much is impossible to say. Similarly, it is impossible to know how many times the e-mails about the survey may have been deleted automatically by recipients who did not make a conscious decision to refuse the invitation to take the survey. During May 2008, along with our survey invitation STC sent out multiple e-mails to members about the upcoming annual conference. Many members in the sample may have paid scant attention to our initial e-mails about the survey because the first identified stc@stc.org as the sender. (We had the STC staff member change the sender to ddayton@stc.org for the two reminder e-mails.)
We believe that most of our survey's nonrespondents were passive, not active nonrespondents. Based on their in-depth field study, Rogelberg et al. (2003) concluded that only about 15% of nonrespondents to organizational surveys were active nonrespondents, and also concluded that passive nonrespondents were identical to respondents when the survey variables had to do with attitudes toward the organization. While our survey was directed at members of an organization, the questions were not about the organization, and the type of organization is a special class—professional membership organizations. Thus, we cannot assume that the findings and reasoning reported by Rogelberg et al. (2003) apply to our nonrespondents; on the other hand, we think the question raised is one worth considering in regard to our survey: Were most nonrespondents passively passing up the chance to take our survey, or were most of them actively rejecting the invitation because of some attitude related to the topic of the survey or attributable to some other cause that might mean that their answers on the survey would be significantly different from the answers of those who responded?
If failing to achieve a certain response rate is not automatically an indicator of nonresponse bias in a sample survey, how then can we estimate the likelihood that the survey results are biased because of missing data from the random sample? Rogelberg (2007) summed up the answer: “Bias exists when nonrespondent differences are related to standing on the survey topic of interest such that respondents and nonrespondents differ on the actual survey variables of interest” (p. 318). Translating that into plain English for the case in question, if a significant proportion of our survey's nonrespondents were significantly different from respondents in their experience with or attitudes toward single sourcing and content management, then their missing data represents a source of bias in our survey results. Thinking about why recipients of our e-mails about the survey would purposely ignore or actively reject the invitation, we surmise that most such active nonrespondents, as opposed to the likely majority of passive nonrespondents, would have found the survey topic of little interest because they had no experience with single sourcing and/or content management systems. Even though we worded our survey invitations to stress our desire to collect information from all STC members, regardless of whether they used SS/CM methods and tools, it seems likely that many recipients of our messages who had no experience with such methods and tools would have felt disinclined to take the time to fill out the survey. To the extent that our conjecture about this is accurate, the survey results would overestimate the proportion of STC members whose work groups used SS/CM methods and tools in May 2008.
References for Appendix B
Curtin, R., Presser, S., & Singer, E. (2000). The effects of response rate changes on the index of consumer sentiment. Public Opinion Quarterly, 64, 413–428. doi:10.1086/318638.
Groves, R. M., Presser, S., & Dipko, S. (2004). The role of topic interest in survey participation decisions. Public Opinion Quarterly, 68, 2–31. doi:10.1093/poq/nfh002.
Groves, R. M., & Peytcheva, E. (2008). The impact of nonresponse rates on nonresponse bias: A meta-analysis. Public Opinion Quarterly, 72, 167–189. doi:10.1093/poq/nfn011.
Keeter, S., Miller, C., Kohut, A., Groves, R., & Presser, S. (2000). Consequences of reducing nonresponse in a national telephone survey. Public Opinion Quarterly, 64, 125–48.
Leslie, L. L. (1972). Are high response rates essential to valid surveys? Social Science Research, 1, 323–334. doi:10.1016/0049-089X(72)90080-4.
Merkle, D., & Edelman, M. (2002). Nonresponse in exit polls: A comprehensive analysis. In R. M. Groves, D. A. Dillman, J. L. Eltinge, & R. J. A. Little (Eds.), Survey nonresponse (pp. 243–258). New York: Wiley.
Radwin, D. (2009). High response rates don't ensure survey accuracy. The Chronicle of Higher Education, (October 5). Retrieved from http://chronicle.com/article/High-Response-Rates-Dont/48642/.
Rogelberg, S. G. (2006). Understanding nonresponse and facilitating response to organizational surveys. In A. I. Kraut (ed.)., Getting action from organizational surveys: new concepts, methods, and applications (pp. 312–325). San Francisco, CA: Jossey-Bass.
Rogelberg, S. G., Conway, J. M., Sederburg, M. E., Spitzmüller, C., Aziz, S., & Knight, W. E. (2003). Profiling active and passive nonrespondents to an organizational survey. Journal of Applied Psychology, 88(6), 1104–1114. doi:10.1037/0021-9010.88.6.1104.
Visser, P. S., Krosnick, J. A., Marquette, J., & Curtin, M. (1996). Mail surveys for election forecasting? An evaluation of the Columbus Dispatch poll. Public Opinion Quarterly, 60, 181–227.
Appendix C: Survey Documents
Link to a Non-Working Archival Copy of the Survey
http://www.zoomerang.com/Survey/WEB22B38UWBJKZ
Copy of Survey Notification Message from STC President Linda Oestreich
Subject: Please participate in a research study of STC members
The STC is sponsoring research to discover the range of information development methods and tools being used by STC members. We especially want to know how many members are using single sourcing and content management methods and tools.
Whether or not you use single sourcing and/or content management, we need your input. You are included in the small random sample of members who will receive an e-mail containing the link to an online questionnaire.
The survey can be done anonymously, or you can provide an e-mail address for follow-up contact or to receive an early view of the results. Most testers reported that they completed the survey in 10 to 15 minutes.
I am excited that Dr. David Dayton (PhD, Technical Communication) and Dr. Keith Hopper (PhD, Instructional Technology) have designed and tested the survey instrument and are ready to collect and analyze the data that you provide.
Look for an e-mail with a link to the survey on Tuesday, May 13.
Dr. Dayton will give a report on the survey results at a session of the 2008 Technical Communication Summit, which will be held in Philadelphia June 1-4.
Copy of First E-mail Message Containing a Link to the Survey
Subject: Please participate in a research study of STC members
We professional technical communicators lack reliable data on the range of information development tools and technologies being used by practitioners.
The STC is sponsoring research to collect that information, with a focus on finding out what single sourcing and/or content management methods and tools are being used.
Your name was among the small random sample of members receiving this invitation to participate in an online survey accessed at this page: [typed here was a link to the informed consent Web page reproduced after this message]
The survey can be done anonymously, or you can provide an e-mail address for possible follow-up contact or to receive an early view of results. The exact set of questions presented will depend on your answers to key questions, so the time required to fill out the survey will vary. Most testers reported that they completed the survey in 10 to 15 minutes.
Whether or not you use single sourcing and/or content management, we need your input. By participating, you will help us construct a reliable profile of information development methods and tools used by STC members.
Because the random sample is a small fraction of the total STC membership, it is critical that we have your data in the survey results. It is equally critical that members of the sample do not forward the survey link to others.
If you have any problems with the link to the survey or with the survey itself, please contact David Dayton at ddayton@rcn.com.
David Dayton: research project lead Towson University (Maryland)
Keith Hopper: survey deployment and statistical analysis Southern Polytechnic State University (Georgia)
Copy of informed consent Web page giving access to the survey
Single Sourcing and Content Management in Technical Communication: A Survey of STC Members
Consent Form
Because you were included in a small random sample of STC members, your information is vital to achieving the purpose of the survey even if you do not use single sourcing or content management. This consent form is required by federal regulations. By clicking the agreement link at the bottom of this form, you acknowledge that your participation is voluntary, that you may abandon the survey at any point, and that your information is anonymous unless you provide contact information, in which case we promise to handle your information with the strictest confidentiality.
Time Required
Most testers of the survey reported that it took them 10–15 minutes to fill out the questionnaire that will appear after you click on the “I agree” link at the bottom of this form.
Purpose of the Study
This survey will collect information from a sample of STC members about their use or non-use of single sourcing and content management tools and methods–and their opinions about them. (In the survey, we define precisely what we mean by “single sourcing” and “content management.”)
What You Will Do in the Study
Your only task is to fill in the Web survey itself.
Benefits
Respondents who complete the survey will be offered an early look at the preliminary data, which we will continue to analyze and will later report in conference presentations and published articles. As a technical communicator, you may benefit in that the survey data will provide a statistical snapshot of the information development methods and tools that STC members are using today and their opinions about some of those methods and tools.
Confidentiality
The information you provide will be handled confidentially. If you choose not to identify yourself to us, we will not try to find out who you are. You will have the option of identifying yourself for follow-up contact by e-mail or to view the preliminary survey results.
We will present the survey findings in terms of group percentages, look for common themes in the open-ended questions, and cite remarks where they are interesting and appropriate. No individual respondents will be identified.
Risks
We do not believe there are any risks associated with participating in this survey.
Voluntary Participation and Right to Withdraw
Your participation in this study is completely voluntary, and you have the right to withdraw from the study at any time without penalty.
How to Withdraw from the Study
If you want to withdraw from the study, you may do so at any time simply by closing the browser in which this form or the questionnaire appears.
Whom to Contact About this Study or Your Rights in the Study
Principal Investigators
David Dayton, ddayton@rcn.com, Towson University (Maryland)
Keith Hopper, khopper@spsu.edu, Southern Polytechnic State University (Georgia)
Chairperson, Institutional Review Board for the Protection of Human Participants, Towson University (Maryland): Patricia Alt, palt@towson.edu
Agreement
If you agree, click here to start the survey. If you experience a problem with the link above, please copy and paste the following URL into your browser: [full Web address to the survey was typed here]
If you do not agree to participate in the survey, please close the browser now or go to the STC home page.
THIS PROJECT HAS BEEN REVIEWED BY THE INSTITUTIONAL REVIEW BOARD FOR THE PROTECTION OF HUMAN PARTICIPANTS AT TOWSON UNIVERSITY.
