By Dawn Armfield, Ann Hill Duin, Isabel Pedersen
A whole host of sectors are incorporating immersive technologies for industrial and everyday use. TechCrunch, ABI Research, and Business Insider predict that the industry of augmented reality (AR) and virtual reality (VR) will reach more than $100 billion by 2020, with Apple, Microsoft, and Google making major investments in AR technologies (Evans and Koepfler).
Smart glasses use voice commands to free hands so workers can leave voice notes or instructions for others. AR is used to create more complex learning and on-the-spot access to information. Browsers use the camera view of smartphones to create a unique interface that blends and sustains “transductive relationships between multimedia, humans, and the environments we inhabit—such that all three of these spheres affect and become affected by one another” (Morey and Tinnell). The composing environment long ago shifted from a fully textual landscape to one that includes visual elements and multimodality. Now immersive technologies incorporate more sensory elements than ever before. In short, technical communication has reached its next threshold for change.
What skills do technical communicators need to develop content for these immersive technologies? Technical communicators must design content to be experienced and thus must begin with a journey mindset. We emphasize on the need for a journey mindset based on our experience in these areas:
- Curation of information on immersive technologies
- Examination of content development for immersive environments
- Expansion of understanding of audience in the design of immersive environments
- Development of a set of heuristics regarding the knowledge and skills needed to work within these environments
Curation
The Fabric of Digital Life research archive, or “Fabric,” monitors the emergence of digital technology prototypes, inventions, news, and research by archiving representations in several categories and media types (text, image, video, etc.). Fabric concentrates on platforms of human-computer interaction to reveal the multiple ways that technologies emerge in society. To understand the depth and breadth of immersive technologies, and most recently, of AR, we curated a corpus of information on human-centered design for AR. As of this writing, these 223 artifacts allow us to see how the landscape of immersive technologies is shifting and how the understanding of our role as technical communicators in this landscape must also shift.
Based on this curation as well as our collective work in designing, deploying, and studying AR and VR devices, we emphasize that technical communicators must engage in participatory design and understand that immersive content demands an experiential mindset. Where once required to be objective and nonexistent in the presentation of information and documentation, technical communicators now must become a part of the process and integrate their own experiences into it.
Content Development
Content development for immersive environments is not new. For example, there has been significant immersive content development in curation and museum work. Some of the foremost museums are using immersive apps, such as Smartify, to increase audience engagement with exhibits. The focus of these types of apps is on the user’s experience within the curated environment.
In medicine, the Federal Drug Administration (FDA) recently approved the use of one AR system, OpenSight Augmented Reality System, in surgical environments. This type of integration holds deep implications for technical communicators: “The technology also allows for a multi-user experience, as multiple HoloLens headsets can be worn among users for training or clinical purposes” (Rohman).
Developing training materials, enhancing clinical aids, or creating museum displays are just a few of the areas technical communicators work in. With these areas moving toward immersive technologies, it is imperative that technical communicators develop the skills needed to work in these environments.
A few years ago, Angie Li and Therese Fessenden writing for the Nielsen Norman Group, described what AR meant for user experience (UX). They argued that AR helps UX in three distinct ways: “1. By decreasing the interaction cost to perform a task, 2. By reducing the user’s cognitive load, [and] 3. By combining multiple sources of information and minimizing attention switches.”
What we understand now is that the implications of immersive technologies on UX or other technical communication areas go even deeper. Technical communicators must be able to draw on their knowledge of immersive environments, participatory design, audience analysis, and content development. By leveraging these skills, technical communicators can embrace a journey mindset.
A Journey Mindset
A journey mindset begins with participatory design. A focus on the role of the communicator as both developer and participant allows technical communicators to have a more integrated connection with content, as well as understanding why and how an audience might respond in certain ways. This is very similar to what technical communicators have always done in creating personas and profiles for usability or other documentation development. However, participating in this way also asks the technical communicator to become a part of the experience—to understand what is at stake from a subjective perspective and why that immersion into the development of the experience is imperative.
In addition to traditional personas, technical communicators must focus on the personas created by the content itself and how those personas in turn affect the audience. It’s one thing to read text and follow instructions. It’s wholly another to have a device connected to your body and to be immersed in the information.
A key component of participatory design is reflection. The concept of empathy, an inherently reflective practice, has been well developed in the context of design thinking. In technical communication, however, there is very little scholarship on the incorporation of empathy into technical communicators’ work.
Empathy is about more than understanding what an audience may think or feel; it is a deeper understanding of why an audience may think or feel in certain ways. The most commonly used definition of empathy is developed from Ralph Greenson’s analysis, in which it is defined as the ability “to share, to experience the feelings of another person.” The concept that a technical communicator actually experiences the feelings of his/her audience is at the crux of a journey mindset.
Another common definition comes from Eva Köppen and Christoph Meinal: “Empathy is the attempt to reconstruct the specific perspective of the other and how he perceives the situation. The aim of empathy is to construe mutual understanding.” Once technical communicators understand how the audience will perceive the situation, they can begin to build better and more adaptive immersive environments.
Therefore, a journey mindset focuses on experiences. Our research in immersive technologies has led us to conclude that audiences in immersive environments are more engaged with the idea of an experience rather than with a straightforward explanation. For instance, the examples of museum curation and medical integration introduce the notion that while the information has always been available to the targeted audiences in the forms of signage, posters, brochures, or books, the experience of immersive technologies allows the audience to focus on the more detailed areas of information.
A museum-goer might be able to learn more about an artist with the touch of a finger to glasses. A medical student could learn that a cut to a certain organ could produce devastating effects and could see those effects without harming another human being. That experience adds to their body of knowledge, developing a deeper connection with their learning.
Sharing design principles for both AR and VR, Bobby Gill emphasizes that “people are seeking out experiences—not technologies” and that we should “shift our mindset to the world of ‘first-person design’ as compared to the third-person experience” and by so doing, that we should be prepared to give up a certain level of control over the experience, that we should toggle between focusing on how to navigate the experience and how to navigate within it, and that we should (of course) “iterate, test, learn, and repeat.”
Our emphasis on a journey mindset was inspired by Robert Hoyle Brown’s Augmenting the Reality of Everything, in which he stresses that the real value of immersive technologies will be realized once we begin to think “of every process—in every industry—in every interaction— as a journey.” Focusing on AR, he writes that “AR can be a personal step-by-step process guide or ‘Sherpa,’ delivering just-in-time precision information, fusing things like training videos, sales spiels, and guidebooks into an intuitive, engaging, measurable, and actionable real-time immersive experience” (p. 8).
An immersive journey includes six elements:
- Flow, the overall actions required for the journey
- Intro, where the journey begins
- Genre, the context for the journey
- Plot, the central story or process
- Vignette, the visual vernacular style to establish the mood
- Outro, the shift back to the real world (pp. 14–15).
It is critical to remember that each audience engages with each element in a unique way.
Heuristics
Figure 1 illustrates a heuristic that we have constructed to guide technical communicators as they embark on this journey mindset. These heuristics are predicated on years of research within immersive technologies, design, UX, and audience analysis.
As part of these heuristics, we include these framing questions:
- Authenticity: What is my audience’s point of view? Is it controlled by the user or independent of the user? Is it meaningful? Is it credible (ethos)?
- Embodiment: How might I position the audience in this augmented place?
- Empathy: Why does my audience think or feel in certain ways?
- Accessibility: Can my audience access or experience the augmented environment?
- Usability: Is the environment usable for various audiences?
- Experience: What journeys do my audiences seek, and how might I augment each experience?
- Immersion: How might the augmentation allow me to create experiences not possible in the physical world? How might I stretch and magnify the audience’s role?
Some of these heuristics may seem antithetical to the role of the traditional technical communicator. Until now, technical communicators primarily have created content for information, consistency, and usability. With immersive technologies, technical communicators require a journey mindset that includes practice of participatory design to identify how individuals connect the content with themselves (identity) and their environment, as well as skills to create content for a journey that is dynamic, inconsistent, highly evolving, and incredibly personal for each participant. Technical communicators must also consider how we, as the designers/creators/participants, shape the content based on practicing empathy and engaging with the journey.
Conclusion
Technology companies are putting vast amounts of money and research into infrastructures for immersive technologies, and the “AR Cloud” will bring immersive content ever more intertwined with the real world. “As we move toward a future of immersive technologies and artificial intelligence, we need to anticipate new ethical challenges and develop practices and policies for a world of deeply immersive experiences” (Craig and Georgieva). This new landscape requires technical communicators to move outside the realm of writing to acquire understanding of various disciplines that help with creating a more informative, engaging, and ethical experience. We are no longer involved primarily in the dissemination of quality information; we are now combining that with the experience of engaging with that information in immersive landscapes. We look forward to the journey.
References
Armfield, Dawn, Ann Hill Duin, and Isabel Pedersen. “Experiencing Content: Heuristics for Human-Centered Design for Augmented Reality.” 2018 IEEE International Professional Communication Conference (ProComm), 241–247. Toronto: University of Toronto, 2018.
Brown, Robert Hoyle. Augmenting the Reality of Everything. Cognizant: Center for the Future of Work. https://www.cognizant.com/whitepapers/augmenting-the-reality-of-everything-codex3050.pdf, 2017.
Craig, Emory, and Maya Georgieva. “VR and AR: The Ethical Challenges Ahead.” https://er.educause.edu/blogs/2018/4/vr-and-ar-the-ethical-challenges-ahead, 2018.
Duin, Ann, Dawn Armfield, and Isabel Pedersen. “Human-Centered Content Design in Augmented Reality.” Context Is Everything, edited by Guiseppe Getto, Jack T. Labriola, and Sheryl Ruszkiewicz. Abingdon, United Kingdom: Routledge, forthcoming.
Duin, Ann Hill, Joseph Moses, Megan McGrath, and Jason Tham. “Wearable Computing, Wearable Composing: New Dimensions in Composition Pedagogy.” Computers and Composition Online, 2016. http://cconlinejournal.org/wearable/.
Evans, Kieran, and Jes A. Koepfler. “The UX of AR: Toward a Human-Centered Definition of Augmented Reality.” UX User Experience, 2017. http://uxpamagazine.org/the-ux-of-ar/.
Gill, Bobby. “7 Things to Know About Designing for Virtual and Augmented Reality.” https://www.invisionapp.com/blog/designing-for-vr-ar/, 2016.
Greenson, Ralph. “Empathy and Its Vicissitudes.” The International Journal of Psycho-Analysis, v41, p. 418, 1960.
Köppen, Eva, and Christoph Meinel. “Empathy Via Design Thinking: Creation of Sense and Knowledge.” Design Thinking Research: Understanding Innovation, edited by H. Plattner et al. Switzerland: Springer International Publishing, 2015.
Li, Angie, and Therese Fessenden. Augmented Reality: What Does It Mean for UX? Nielsen Norman Group. https://www.nngroup.com/articles/augmented-reality-ux/, 2016.
Morey, Sean, and John Tinnell. Augmented Reality: Innovative Perspectives Across Art, Industry, and Academia. Anderson, SC: Parlor Press, 2017.
Rohman, Melissa. “FDA Approves First Microsoft HoloLens Augmented Reality system for Surgical Use.” Health Imaging: Insights in Imaging and Informatics. https://www.healthimaging.com/topics/advanced-visualization/fda-approves-augmented-reality-system-microsoft-hololens, 2018.
Tham, Jason, Ann Hill Duin, Laura Gee, Nathan Ernst, Bilal Abdelqader, and Megan McGrath. “Understanding Virtual Reality: Presence, Embodiment, and Professional Practice.” IEEE Transactions on Professional Communication. 61:5 (2018): 178–195.
Resources
Canon, Sarah, and Ann Hill Duin. An Introduction to Augmented Reality. Minneapolis, MN: UMN Digital Conservancy, 2018.
Fabric of Digital Life Research Archive, The authors’ corpus of information on human-centered design for augmented reality, https://fabricofdigitallife.com/index.php/Browse/objects/facet/collection_facet/id/20
Smartify, https://smartify.org/
Tham, Jason, Ann Hill Duin, Laura Gee, Nathan Ernst, Bilal Abdelqader, and Megan McGrath. “Understanding Virtual Reality: Presence, Embodiment, and Professional Practice.” IEEE Transactions on Professional Communication. 61:5 (2018): 178–195.
DAWN M. ARMFIELD (dawn.armfield@mnsu.edu) is an Assistant Professor of Technical Communication in the Department of English at Minnesota State University, Mankato.
ANN HILL DUIN (ahduin@umn.edu) is a Professor of Writing Studies at the University of Minnesota where her research focuses on emerging technologies.
ISABEL PEDERSEN (Isabel.pedersen@uoit.edu), Canada Research Chair in Digital Life, Media and Culture, is an Associate Professor at the University Of Ontario Institute Of Technology in Canada.
