By Raymond Gillespie | Senior Member
I first heard the term “technical debt” a few years back during a training course on Agile software development. It refers to a “knowledge debt” that can build up in long-term software development projects. The term was originated over two decades ago by the respected software engineer and Wiki pioneer Ward Cunningham. Nobody explains the concept better than Cunningham himself:
If you develop a program for a long period of time, by only adding features, and never reorganizing it to reflect your understanding of those features, then eventually that program simply does not contain any understanding. All efforts to work on it take longer and longer. In other words the interest [on the debt] is total.
Now, reread those words, but this time, substitute the phrase “documentation set” for “program.” Could this describe a documentation project that you have heard about, or come across, or—heaven forbid—worked on?
In my case, the answer is yes. And from conversations with other technical communicators, I know that I am not alone in having worked on documentation that seems to be nearing the point where it “eventually … does not contain any understanding.”
Debt: A Useful Metaphor for Documentation
Cunningham’s inspiration for the debt metaphor was the work of the linguists George Lakoff and Mark Johnson who argue, “the way we think, what we experience, and what we do everyday is very much a matter of metaphor.”
By using the debt metaphor, Cunningham found he could better explain to his boss why his team was struggling to understand the complex financial software they were developing. In short, they had a knowledge debt. And there was interest to pay on that debt—in terms of time and energy to regain their understanding—if they were to resume making progress.
The debt metaphor also enabled Cunningham to propose a solution. By refactoring the software—by studying it, understanding it, and restructuring it—they aimed to “make [the code] look as if we had known what we were doing all along.” Cunningham had in effect come up with a debt repayment plan. A plan that would allow them to gradually reduce the knowledge debt, gradually improve the structure of their software, and gradually reach a point where the project could again move forward from a position of understanding.
I would argue that the technical communication profession could adopt the debt metaphor to help us understand and talk about knowledge gaps in our complex projects. The question is, can we borrow some of the debt repayment ideas pioneered by our engineering colleagues and apply them to indebted documentation? Can we, in a similar way to them, refactor our documents to make them look as if we had “known what we were doing all along?” We will return to this question later on. First, let’s look at some causes of debt in documentation sets.
How Does Technical Debt Accrue in Documentation?
There are many possible reasons why debt can creep into a documentation set. Here are three that I have witnessed.
1. Documenting New Features Only
In this day of tight deadlines and lean documentation teams, we have to budget our writing resources carefully. We have to prioritize, and it usually makes sound business sense to prioritize the documentation of new features. But this comes with a risk: it can lead to a chronic neglect of previously written documentation.
For example, in one project I worked on, new product features—the features which generated revenue—were provided to the customers in “minor software releases” every three or four months. For each new feature, we wrote a description of that feature, as well as instructions for its installation, activation, and usage. The new documentation was well written, well tested, and well received by our customers. However, we often postponed the update of older documents to reflect these new features. This work was instead scheduled for the next major release. By the time this big release came round, the backlog of work for the older documents was substantial, our knowledge resources insufficient. With each release we were sinking deeper into debt.
2. Increasing Complexity of Projects
Most technologies, and thus most software products, get more complex overtime. Take, for example, the field of mobile telecommunications. I was first involved with a telecom project in the late 1990s. At that time, mobile networks were relatively simple 2G networks. Since then, we’ve had 2.5G, 3G, and 4G—each generation introducing its own forest of standards, its own new technologies, its own challenges for remaining compatible with the generations that preceded it. Understandably, it is becoming increasingly difficult for technical communicators in the telecom field to keep on top of this complexity. Some degree of technical debt is inevitable in the documentation of a large telecoms product. I would also hazard an informed guess that keeping up with accelerating complexity is one of the biggest challenges facing documentation teams in many other fields.
“Software systems almost always degrade into a mess.”
—Robert C. Martin
3. Knowledge “Churn”
When an experienced writer leaves a project, knowledge is lost. Similarly, when a documentation project is outsourced, or insourced, or moved in-house to a different team, knowledge is lost. Some loss of knowledge is inevitable for any moved project, regardless of its age, its complexity, or how conscientious the previous team has been in following the “knowledge transfer” process.
When knowledge is lost from your project, you just have to hope it won’t tip your documentation into debt. If it does, you will have interest to pay in terms of time, money, and intellectual energy to get back into the black.
How Do We Get Out of Debt?
Let’s return to the question of whether we can apply the refactoring techniques used by our software engineering colleagues to the domain of technical documentation.
Read any textbook on the topic and it soon becomes apparent that automated testing is the key to successful software refactoring. By having a comprehensive set of regression tests that can be run nightly, daily, or even hourly, the engineering team can quickly see if any of their refactoring efforts have caused a test to fail. Thus, they can get on with tidying up the software, making it more understandable, without having to worry about “breaking the code.”
“There are no easy answers. There is no such thing as a best solution.”
—Andrew Hunt and David Thomas
Unfortunately for technical communicators, documentation normally doesn’t lend itself to automated testing. So we have to get creative when looking for a means to safely restructure a particular set of documents.
To follow are a few example actions I’ve seen help reduce debt in problematic documentation projects I have been involved with. I hope that they will give you some ideas for preventing and curing debt in your project.
Get Close to the Software Testers
Software testers may be called any one of several names in your organization—quality assurance, system integration, functional testing. Regardless of their title, it can pay dividends to get close to the group that tests the functionality of the software. This is the team that tests to make sure that the software does what we’ve promised the customers it will do. To design, run, and analyze suitable tests, the testers need to fully understand not only what the software should be doing, but also how to install it, configure it, and run it. In other words, the very same things that we are writing instructions for in our documents.
In one project I was involved with, our writing team was lucky enough to be physically located amongst the testing team. We attended their test plan reviews, their training sessions, and even their daily stand-up meetings. This proximity gradually created a great working environment with advantages for both the testers and the documentation team. For example, testers began to correct errors in the documents that came up during testing, and even—with guidance from the writing team—became motivated to write the initial drafts of some operating procedures. But perhaps the biggest unforeseen consequence of this symbiosis was that the knowledge of the writing team—in terms of both technical and product knowledge—started to grow exponentially. And with this knowledge, the team began to have greater assurance when it came to editing and restructuring previously written “legacy” documentation. In short, the team, without any prompting from upper management, began to spontaneously refactor its documents.
Get Close to the Customer Support and Software Maintenance Teams
For 14 months, I had the opportunity to work as a member of a software maintenance team. It was one of several teams supporting a complex telecom project for which I had previously been writing documentation. I found it a deeply educational and eye-opening experience on many levels. One of my tasks was to analyze customer trouble cases that had been escalated from the technical support team. From a documentation viewpoint, what jumped out at me was how few cases were due to errors in the documentation. However, there was a significant number of trouble cases due to:
- misinterpretation of overly complex or scattered instructions
- inability to find information
- looking at the wrong version of a document
In other words, problems that should not occur in a logically structured document set.
However, I observed that cases such as these—each one an indicator of the need to refactor the documentation—were seldom communicated to the customer documentation team. The problem was that the technical support and software maintenance groups were under great pressure to resolve customer issues quickly. There was a necessity to “fix” each issue as soon as possible and move onto the next one. Sending improvement requests to the documentation team was seldom seen as a priority.
If the situation in your organization is similar, I would recommend that you, as a technical communication professional, take the initiative to forge a communication channel with the technical support teams. Do whatever you can to get close to these busy colleagues, to extract precious information from them without adding to their already substantial burden. At the very least, make sure that you and your team members all have user accounts to whatever trouble ticket system the support team is using. Check the tickets regularly, look out for information-related cases you can help with, and learn about the issues that are affecting your customers. Then take the time to reflect on what you’ve learned and ask yourself how you can best use that knowledge to refactor your document set.
3 Ideas for Tackling Document Debt
- Get closer to the software testers
- Get closer to the customer support and software maintenance teams
- Use a topic-based writing approach
Stop Writing Books, Start Writing Topics
I recently met an ex-colleague who is working on the same project that I mentioned earlier; the project that overly concentrated on new features. He had good news for me. They have turned the corner and are now gradually getting the documentation out of debt. How are they doing this?
It turns out that they have recently shifted toward a topic-based writing approach, namely DITA. They are in the process of breaking documents down from their previous “big book” structure into small topics. This dissection has had an unexpected benefit in terms of their knowledge acquisition. It has forced the writers to think deeply about the meaning and structure of their documents. It is making them ask questions: Can I split this procedure into smaller task topics? Should I put this part into its own concept topic? Do I need this piece of information at all?
What is more, members of the team, who previously worked solo on their own big documents, are now talking with each other, looking for opportunities for reuse of topics and removal of redundant information.
With each question answered, with each discussion resolved, the knowledge of the team is increasing. Topic by topic, the debt in the documentation set is being paid off.
Conclusion
These are challenging times for technical communicators. Software and hardware are getting more complicated; documentation teams are getting smaller and increasingly bereft of experience. We need to recognize that these factors can lead to a “lack of understanding” creeping into our complex documentation sets. Cunningham’s debt metaphor gives us a simple means to communicate about that lack of understanding. It is a metaphor that can be quickly grasped by other writing team members, by our development colleagues, and by our managers. Only by acknowledging that a knowledge debt exists in our documents, can we take stock of that debt and put together plans to repay it.
Raymon Gillespie (raygillespie@yahoo.com) has 20 years experience in the field of information technology. For the past 14 years, Raymond has lived in Budapest, Hungary, working as both a software engineer and a technical writer, mainly in the fields of telecommunication, medical imaging, and navigation software. He holds an MSc in information management from Lancaster University Management School (UK).
References
Cunningham, Ward. 2009. “Debt Metaphor.” YouTube.com, accessed 3 July 2014, www.youtube.com/watch?v=pqeJFYwnkjE.
Hunt, Andrew, and David Thomas. 2000. The Pragmatic Programmer. Addison Wesley Longman. Kindle edition.
Lakoff, George, and Mark Johnson. 1980. Metaphors We Live By. University of Chicago Press. Kindle edition.
Martin, Robert C. 2005. Foreword to Working Effectively With Legacy Code, by Michael Feathers. Pearson Education.
