How do we determine the impact of research? I almost used the word measure, but that would imply and privilege a quantitative standard of success. Although calculations of impact are more and more ubiquitous in journal publication, I think we are wise to recognize their limits and give equal or greater weight to qualitative judgments of influence.
Note that a variety of formulas for calculating “impact” are available:
- Impact Factor – measures the impact of a journal on a field by looking at how many articles it publishes in a two-year period that could be cited and using this figure to divide the number of citations of these articles in following years. This measures the journal’s impact instead of the impact of individual articles, but it is assumed that high impact journals are more likely to publish high impact articles.
- H-index – measures a given author’s impact on a field by looking at the author’s most frequently cited articles and finding the number of times these articles are cited by other scholars in their publications. This measure again offers indirect evidence of the influence of individual articles, as it is assumed that high impact authors are more likely to publish high impact articles.
- Citation Analysis – measures the impact of a given article by looking at the number of times it is cited by other scholars in their publications. The results of a citation search will differ according to the database it uses (e.g., Web of Science versus Google Scholar).
- Views/Downloads – measures the impact of a given article according to the number of times it is viewed or downloaded. This assumes that accessing of the article is itself evidence of its influence. It privileges the research decisions of scholars who have access to the journals in question as well as journals that allow unlimited access to their articles.
- Social Media References – measures the impact of a given article according to the number of times it has been cited, discussed, liked, or shared through social media. This calculation may be qualified by giving greater weight to influential sources (e.g., prominent scholars) but assumes active participation in social media, thus privileging scholars with unrestricted/unfiltered access.
Although the several metrics regarding impact are impressive in their perceived objectivity and could have merit for comparative purposes, we also must recognize that each is a portrait of impact built on incomplete records of citations (either formal or informal) as well as inconsistent and inadequate evidence of influence.
I have cited scores of articles through the years that were passing contributions to the field but were pertinent to the manuscript I was writing at the time. Meanwhile, I have also read a lot of journal articles that have influenced my teaching and research—the way I think about the field—but that I have never considered necessary to cite in my articles and books, because their impact was indirect and abiding instead of direct and immediate.
For example, Kathleen Durack’s “Patterns for Success: A Lesson in Usable Design from U.S. Patent Records” (Technical Communication, 1997, pp. 37–51) changed my vision of the field of technical communication, encouraging the analysis of a wider variety of artifacts as technical communication. I have, however, never been given cause to cite this article in my research projects. Neither have I cited Saul Carliner’s “Modeling Information for Three-dimensional Space: Lessons Learned from Museum Exhibit Design” (Technical Communication, 2003, pp. 554–570), though I know it has immensely influenced my thinking on usability, accessibility, and information design.
I would imagine that each of us could point to important articles of persistent influence, articles that we have appreciated—and might have praised in e-mail messages, in office conversations, in classroom discussions—articles that have inspired us and guided us but that we have never cited in subsequent studies, or downloaded, or mentioned in publicly available digital spaces.
For example, at my invitation, all of the authors in this issue of the journal discussed the major influences on their research projects. Their explanations make clear that impact resists simple definition or numerical scoring.
“A One-Hundred Forty Character Discourse: The Twitter Apology as an Emerging Sub-Genre of Corporate Communication,” by Allen Berry, analyzes 40 corporate apologies issued through Twitter to determine the rhetorical techniques contributing to their success or failure. The analysis focuses on offensive corporate activity that has been identified and discussed on Twitter and discovers that effective apologies by the guilty parties almost always recognize the error or injury and express remorse, usually acknowledge responsibility, often propose to make restitution, and sometimes specifically promise to avoid repetition of the wrongdoing. Vivid with examples, the analysis also makes clear that sensitivity to Twitter’s wide international audience as well as swift and immediate action are essential to this new articulation of apologetic rhetoric.
Allen’s project started with his readings in a post-doctoral independent study on the subject of social media and conversations with his professor and mentor, Dr. Ryan Weber. Allen readily identifies the research of highest impact:
When I think of the literature that bore the greatest influence on the article, there are two that immediately come to mind. Mainly, John Kador’s book on apology strategies, Effective Apology: Mending Fences, Building Bridges, and Restoring Trust (2009). It was simple and elegant, and it put the strategy in words that were easy to understand and embrace. He offered examples ranging from corporate apologies to an apology on the part of the United States Military to a group of Muslim clerics after a group of soldiers used a copy of the Koran for target practice. In addition, Ruth Page’s article, “Saying ‘sorry’: Corporate Apologies Posted on Twitter” (2014), was interesting to me. Her information was very similar to Kador’s, but Page was more scholarly and scientific in her approach. Her identification of the actual apology as an Illocutionary Force Indicating Device was very helpful in clarifying that an apology is an overt act. Her article framed the apology as something interactive and viable that requires effort on the part of the offender.
Allen cites 40 sources in his article, but the impact of Kador (2009) and Page (2014) differs obviously from the influence of the remaining 38. He mentions Kador 37 times in the article and Page 14 times, but Allen’s words here give us genuine insight on their relative impact.
“The Burden of Ambiguity: Writing at a Cooperative,” by Avery Edenfield, reports on the language practices of a democratically organized business in a two-year study that involves interviews, observations, and analysis of a wide array of written artifacts (e.g., e-mail messages, handbooks, inventory lists, screen shots). This multi-layered examination reveals how ambiguous language operates positively and negatively within this organization: For example, ambiguity in job descriptions creates confusion and paralysis, whereas ambiguity in the bylaws of the organization allow it critical agility during a financial crisis. Ambiguities also must be interpreted, negotiated, and navigated, thus constituting a cost of time and energy for the organization that must be weighed and justified.
Avery explains the influence of earlier studies on this project:
Some of the works that inspired me to think closely on the intersections of writing and power are Carolyn Miller’s “Genre as Social Action” (1984), Bernadette Longo’s Spurious Coin (2000), and Dorothy Winsor’s Writing Power (2003), just to name a few. The works that guided me on thinking about democratic organizations and research are all over the place, but I particularly was inspired by Francesca Polletta’s Freedom is an Endless Meeting: Democracy in American Social Movements (2002), George Cheney’s “Democracy in the Workplace: Theory and Practice from the Perspective of Communication” (2009), Jo Freeman’s “The Tyranny of Structurelessness” (1972), and Teresa Harrison’s “Communication and Interdependence in Democratic Organizations” (1994). Also, as of late, I’ve drawn inspiration from notable TPC articles on social justice, like Natasha Jones’s “Technical Communicator as Advocate” (2016) and “Disrupting the Past to Disrupt the Future: An Antenarrative of Technical Communication” (2016) by Natasha Jones, Kristen Moore, and Rebecca Walton.
Neither Polletta’s book nor Freeman’s article—two of the nine key influences that Avery identifies—are cited in the article. As I noted earlier, a list of citations is incomplete and insufficient evidence of impact.
“Interactivity in an Age of Immersive Media: Seven Dimensions for Wearable Technology, Internet of Things, and Technical Communication,” by Jason Tham, offers a review of the existing research on interactivity, applies the resulting insights to immersive environments, and proposes guidelines for technical communicators engaged in immersive experience design. This study finds effective interactivity is contingent on the attention given to seven issues (i.e., reciprocity, synchronicity, connectedness, navigability, user control, entertainment, and sensory stimulation). It also encourages technical communicators to focus their efforts on assuring seamless connections for versatility and compatibility of devices, optimizing user control and customization, developing context-sensitive and anticipatory user assistance, and fortifying the immersive experience with more tactile interactions.
While Jason lists 88 sources of information in his article, he mentions three key influences on his thinking about the project, only two of which he cites in the article:
I started researching mediated interactivity in a master’s seminar on media convergence in 2013. A resource I found most useful was The New Media Theory Reader, edited by Robert Hassan and Julian Thomas (2006). However, while I found that the conversations around the development of interactive technologies are rich in computer science, mass communication, and psychology, resources were scarce for technical communication practitioners. Further study has brought me to two landmark articles in Technical Communication that I cite as motivation for this publication––Andrisani et al. (2001) and McDaniel (2009)––which have given me exigence to update the literature on interactivity for technical communication purposes. To anchor my discussion of new interactivity features and dimensions, I ledged onto wearable technologies and the Internet of Things, both emerging technologies that I have been using and investigating since my PhD program.
“Toward Understanding Important Workplace Issues for Technical Communicators,” by Clinton R. Lanier, reports on a survey of practicing technical communicators regarding their perceptions of the key changes in the field that have occurred in the last five years, from adoption of new responsibilities and technologies to adaptation of existing theories and techniques. While the principles of clear communication are still a solid foundation, the survey uncovers evidence of a transformation in the field, especially in the shift to responsive design of multimedia information for mobile devices. This finding itself might be unsurprising, but the survey proves that it is a change neither isolated nor radical but of impressive scope and intensity.
According to Clint, his research project was driven by a sense of obligation to his students and informed by a wide array of resources but especially a 2015 article in Technical Communication:
I believe that it’s the duty of a technical communication instructor to stay informed about the workplace their students will be entering. Unfortunately, that’s not always easy to do. Even with all the great articles that have been published in the past few years, I felt that I still needed a better understanding of what TC professionals cared about, worried about, or felt was important. Eva Brumberger and Claire Lauer’s “Technical Communication as User Experience in a Broadening Industry Landscape” was particularly important. I published a similar article in 2009 and really appreciated their updated information, but it also made me realize nobody had attempted to reach a large pool of actual practitioners, and that’s what I tried to do next. Luckily, social media presents a great platform for reaching these professionals, and I was able to get the opinions of technical communicators from a wide range of backgrounds, training, and professions, and found a wealth of surprising points, including that many of the tools, skills, and concepts I used as a technical writer back in the early 2000s were still considered important and necessary to know.
“An Integrative Literature Review of Project Management in Technical and Professional Communication,” by Benjamin Lauren and Joanna Schreiber, analyzes 128 published materials (chiefly articles in journals and magazines) on the subject of project management in order to determine how project management is described as a practice as well as which theories of communication it studies and which methods or strategies it adopts. The analysis finds that project management is typically perceived as a generic skill instead of a rhetorically sensitive practice that is continuously adapting to each organization and the individuals involved in each project. Important opportunities for critical studies of project management are thus available to technical communicators, and Ben and Joanna seize a salient opportunity by guest editing the forthcoming May 2018 issue of Technical Communication on Project Management.
Ben and Joanna acknowledge the several key influences that guided their thinking:
When we first started talking about the project, there were at least three things we were thinking about. The first was Stan Dicks’ observation in “How Can Technical Communicators Manage Projects?” (2013) that very little scholarship has been dedicated to the topic of project management even though it is essential to technical communication as a field. We immediately wondered, “Well, what has been published?” We had also read Rebekka Andersen’s and Tatiana Batova’s “The Current State of Component Content Management: An Integrative Literature Review” (2015) and were really inspired. They managed to bring together so many different voices into conversation and across what are usually relatively siloed groups. Third, after the annual conference of the Association of Teachers of Technical Writing in 2015, we had a good discussion about how studies of project management would be enriched if we could put academic and industry voices into conversation. This is partly because we came to see that project management methodologies influence the kinds of workplace experiences that technical communicators both participate in and must shape for others. We were also surprised by many of the existing publications on project management. We did not expect the focus on processes and tools instead of critical discussions or reflections on how project management processes and tools are designed systems that determine how we collaborate and participate at work. We felt earlier studies (e.g., Jimmie Killingsworth & Betsy Jones, “Division of labor or integrated teams: A crux in the management of technical communication?,” Technical Communication, 1989) had made this point pretty clearly, but it somehow became less present in more recent publications.
The impact of research, however, is never a judgment made exclusively by the readers using the research. The authors of articles also contribute to the determination of their impact by setting the standards of success.
For example, Allen hopes his article on Twitter apologies will encourage corporations to adopt more effective practices in crisis communication:
The standard operating procedure for dealing with a public scandal calls for the offender to take no responsibility for the offense and attempt to telegraph culpability onto another party, including but not limited to the offended. I hope that my article serves to inform social media managers and others about how to better manage Twitter scandals. While it is virtually impossible to mend the damage with every Twitter user, the damage to a company’s reputation can be reduced, and their reputation even bolstered if the crisis is handled correctly. If the effort is made and made properly, even audiences as diverse and jaded as the exponential Twitter audience are willing to forgive those who offer a sincere apology for their offenses. I hope that this article will thus enhance crisis management strategies. There is an effective strategy for handling a social media apology, and given the results of my study, I hope that corporations will change the old strategy of denying all responsibility in favor of the more elegant strategy described in my article.
Avery believes that his article might encourage a more judicious evaluation of ambiguity’s positive and negative traits in technical communication, a position he arrived at while working on this project:
The first draft of this article examined a variety of genres and their social effects within a cooperative that I had studied. In the initial draft, I argued their documents designated roles and allowed for reporting of violations, but ambiguity undermined many of the documents. Over time, I saw this last feature of ambiguity as a more important piece of the puzzle. When I set out to write the second draft, I didn’t anticipate finding ambiguity to be a positive feature. That was a real surprise to me. It’s easy to assume that clarity and objectivity are always essential elements to effective communication, especially in regulatory documents. However, my analysis pointed to a situation where that was not the case. In the final version of the manuscript, I argue democratic organizations must carefully consider the positive and negative consequences of ambiguity, and that this consideration should extend to regulatory documents where clarity is often an assumed objective.
Jason hopes that his article about immersive media will change design practice as well as teaching:
As I continue to pursue questions and solutions related to user experience and technological development in these emerging technologies, I see great potential for their deployment in education and training, professional practice, and service to the public. Thus, I continue to advocate for critical investigations of these technologies across personal, social, professional, and civic contexts. Ideally, I would like designers and developers to use the dimensions I have devised as guiding principles for devising their own manifestos in technology design. I hope that in every design decision they make they would return to these dimensions and consider the impact on users. For teachers and scholars, I hope to demonstrate how our scholarship can shape future technological actions.
Clint believes his article on workplace practices will lead to a better sense of the field by instructors and their students as well as by practitioners:
I think ultimately this study and the associated findings will really help me and other instructors better educate and prepare students. As for practitioners, I think it will help them understand the direction of the discipline and how their own workplace connects to other professional organizations.
And Ben and Joanna hope their article will generate more studies of project management:
We hope that our article will motivate technical communication to take up critical discussions of project management processes and tools. The scholarship shows we tend to rely on existing processes and tools without interrogating how useful or inclusive they are. A good example is the widespread adoption of Agile, Lean, Design Thinking, and so on. These are development methodologies, yes, but they also dictate how we manage projects and influence the very experiences we produce. While we found that industry thought-leaders often don’t hesitate to adapt project management methodologies to different contexts and situations, that practice does not seem as evident in classroom instruction. We see room for technical communicators in those discussions, offering rhetorical concepts as a way to critically adapt or develop project management methodologies for organizations and teams. This project made us realize how much work is left to be done in project management, and we believe there is a real opportunity for technical communication as a field to participate in these conversations.
While Technical Communication enjoys high impact scores (e.g., Impact Factor of 2.1 in 2016), we must also compile the declarations of authors and the tributes of readers if we are to know the genuine influence of the articles that it publishes. This collaborative effort would give us more compelling information about impact and would certainly be more interesting reading.
And, thus, my questions: Which of TC’s articles stick in your mind? Which have changed your thinking? Influenced your practice? Guided your teaching? Inspired your research?
