Leo R. Lentz, Henk L.W. Pander Maat, and Ted J.M. Sanders
Abstract
Purpose: This paper introduces the Knowledge Base Comprehensible Text, a digital resource containing 702 studies on comprehension and usability of text and discourse, published between 1980 and 2010. The paper explains which publications were included in the knowledge base, how they were collected, how they were annotated and what the interface of the knowledge base looks like.
Method: Literature search
Results: The paper presents a brief survey of the studies contained in this resource, and discusses the answers of the knowledge base to an exemplary question about the comprehensibility of passives.
Conclusion: The knowledge base makes research on comprehension accessible that is relevant for both practitioners and researchers in the field of technical communication. It is a useful tool in a field which, by its very nature, has a strong interdisciplinary orientation and is therefore hard to oversee.
Keywords: empirical evidence, knowledge base, writing, literature search, comprehension, readability, usability
Practitioner’s Takeaway
- Practitioners and students can easily find the results of studies into comprehension effects of message features and discourse genres.
- Professionals can validate practical writing advice by investigating results of empirical studies into related text features—such as the use of passive voice—and thus strengthen their writing and consultancy.
- Students and academics learn to oversee the interdisciplinary field of comprehension research in specific domains, such as health or education, and genres like manuals, forms or patient information leaflets.
- Students in communication studies learn how to perform a systematic and extensive literature search.
Introduction
Comprehensible language—or rather the lack of it—is one of the top 10 bottlenecks for organizations in serving clients and citizens. Banks, hospitals, national and local authorities, and numerous other organizations work at improving the comprehensibility of their information. But what makes a document comprehensible and usable? Researchers have published hundreds of studies to answer that question in different disciplines, such as psychology, psycholinguistics, discourse studies, and document design. Unfortunately, the gap between academic research and everyday practice will keep growing as this body of research expands and practitioners experience stronger time constraints. Expert practitioners have more than once pointed out that they would like to see a firm research base for their work. Palmer and Killingsworth (2002), for instance, report that experts in document design feel a need for more time to study the research literature. A well designed knowledge base could supply a crucial component of the ‘technical communication body of knowledge’ envisaged by Coppola (2010).
In 2007, The Society for Technical Communication started to develop a Technical Communication Body of Knowledge (TCBOK) to help professionals find the knowledge they need. The TCBOK portal offers a large body of knowledge on academic programs in technical communication, marketing tools, career paths, and a research section with an area that contains overviews of research on particular topics, such as Style, Understanding and Memory. A closer view reveals that the knowledge base is limited in reviewing relevant research. An annotated bibliography in that section offers six papers on readability formulae, segmentation, and plain language. Although helpful for those who begin to orientate in that field, this portal shows that technical writers and students cannot rely on this bibliography when they want to oversee the field. In this paper we present a Knowledge Base for Comprehensible Text that makes a large body of research accessible to practitioners, students, and academics. This knowledge base is complementary to TCBOK: it does not inform practitioners on all practical and professional issues like TCBOCK does, but it focuses on the crucial question what makes documents comprehensible.
Since the 1980s, text comprehension has become an interdisciplinary field in which discourse researchers cooperate with cognitive or educational psychologists and human factors specialists. Apart from experimental comprehension research, applied researchers have contributed studies on comprehension in fields such as health communication and government-citizen communication. As a result, comprehensibility research, once begun in the readability paradigm, diversified considerably. It is already hard to list the core journals in the field; a gamut of text features has been investigated, defined differently across various studies; many different measures have been used to gauge comprehension; many studies are traditional experiments manipulating just one or two factors, while revision studies compare original documents with documents that have been modified in many ways.
The lack of transparency of the field is felt by practitioners and students in language and communication. Even after taking a course on text design and comprehension, bachelor students have trouble locating empirical comprehension studies for a given text genre or text feature, in spite of the fact that powerful search engines abound. Those search engines can only be started by entering appropriate search terms. Finding the right terms requires prior knowledge of the research landscape. Both students and practitioners experience a lack of prior knowledge to efficiently use these search engines. This Knowledge Base has been built for both groups.
This paper introduces the Knowledge Base Comprehensible Text (www.comprehensibletext.com) and discusses the selection of studies and the annotation procedure. We provide an overview of the contents of the knowledge base and report on an exemplary excursion. For this excursion we were inspired by the personas introduced in TCBOK. We created the following persona.
Kate Patel, a technical writer with a long-standing career, had a discussion with her contractor about a manual she just produced. Some sentences in the manual used passive voice, a deliberate stylistic choice of Kate. The contractor told her that all sentences had to be written in the active voice. As a general rule, Kate agreed, but not for these particular sentences. The debate could not be solved. Kate starts searching for empirical evidence sustaining her claim that under specific conditions passive voice should be preferred, or at least does not harm.
Before we present the results of her query, we will first discuss the design of the knowledge base and the criteria we used for the selection of studies that can be found in it.
The Design of the Knowledge Base
The development of the knowledge base started with formulating design specifications. The following points of departure were chosen for the design process.
General purpose of the knowledge base: The knowledge base provides systematic information on the studies of the past three decennia concerning message features affecting comprehensibility. The studies must be accessible in terms of text genres, message features, comprehension measures and their combinations.- Language: We produced both English and Dutch versions of the knowledge base.
Target groups: The knowledge base aims both at academic user groups (students, teachers, researchers) and at professional user groups (communication professionals, communication managers).- Prior knowledge: The knowledge base requires some methodological and linguistic knowledge. Hence we use terms like type-token ratio. A glossary is provided for novice users.
- Search routes: There is a fast search option accessible from the home page and an ‘advanced search’ option one click further away. The fast search option offers genre group, feature group and comprehension measure. Advanced search offers more detailed categorizations of genres and features, as well as additional information (for example, on participants and stimuli). Both the fast and advanced search facilities offer a refine search option, in which further fields can be used to narrow down the results.
- Closed and open fields: Most search fields provide dropdown boxes with clickable items, but in the author, title and abstract fields, search strings can be entered freely.
- Combining search terms: When combining terms from different fields, only ‘AND-combinations are facilitated. For instance, combining ‘word difficulty’ with ‘health’ gives studies of word complexity in health texts. In contrast, the system considers combinations of terms from the same field as ‘OR’ combinations. Hence combining ‘word difficulty’ with ‘text lay-out’ yields studies on word difficulty, lay-out, or both.
- Accessibility of the studies: A Google Scholar link is presented for studies that are digitally available. The actual access to the study may of course require a subscription, and hence depends on the work station of the user.
Figure 1 shows the final version of the home page, providing introductory text and the fast search option.
Suppose, our persona Kate focuses on the second option Feature and selects Sentence level features because she wants to find out what research tells us about the use of passive voice. Figure 2 shows how the results look like.
This search yields 93 hits, the first two of which are German and Dutch studies. In the right upper corner, the user may refine the search or create a list of references in a Word file. The last button enables Kate to reorder the list. The refine option leads to the advanced search screen, which offers 16 search features: author(s), title, abstract, year of publication, participant age, genre group, genre, feature group, feature, writing advice, comprehension measures, reading process measures, non-textual independent variables, control variables, modality (spoken, written, digital) and type of stimuli. Combining these terms opens up endless possibilities.
Our persona Kate will decide to refine her search, choosing for the genre of manuals and the sentence feature passives (Figure 3).
This decision to refine the list of results leads to a disappointing “no results.” Probably no research has been done on effects of passive voice in manuals. Thus, Kate decides to look for studies on passives without any genre restriction, which leads to a list of 15 studies (Figure 4).
Clicking on a result produces the data for her query. First, the bibliographical data, the Google Scholar link and the abstract are presented (Figure 5). More information about this paper is presented in Figure 6, such as the advice for which this study is relevant and its conclusions (based on the final section of the article). Finally, further information is presented about the student participants in this study, the kind of document manipulation (isolated fragments) and other, non-textual variables. Kate will notice that the first study in the list advises to avoid passive voice. She probably will look at the other 14 results and find out that some studies report other findings.
This is how the knowledge base works. Now we move to the principles behind the set-up of the Knowledge Base: How did we select the studies? And how did we annotate every study? These questions will be answered in the next two sections. Finally, we will return to Kate and discuss the results of her case study on passive voice.
Selection Criteria
In our literature searches, we aimed to find studies on the effects of textual and visual features on comprehension. The textual features included word and sentence level issues, information order and headings, but also layout issues like the continuous or bulleted presentation of lists and the length of lines. The overwhelming majority of the papers turned out to be reports of experimental studies. But we also found studies in which authentic texts had been revised and tested for comprehension. Studies on visual features focused on comprehension effects of tables, diagrams, graphs, and illustrations.
In our selection, we did not evaluate the quality of studies, as we wanted to survey the field in the broadest possible terms; hence only thematic requirements were used to discard studies. We excluded papers that did not test different message versions on comprehension, such as studies into reading comprehension skills, studies that diagnose and revise but did not test the revisions, studies that investigated information preferences of participants, or studies of reading strategies or reading speed. Some studies using substantial content manipulations were excluded, but studies focusing on the effect of inserting short clarifications have been included.
Studies of the effects of reader variables (for example, age, prior knowledge) have only been included when these variables were crossed with message variables. Studies have only been included when they used documents addressing non-expert audiences, but educational texts addressing novice students (age above twelve) were accepted. Research with younger readers often focuses on the development of reading comprehension skills, a topic that is less central to our purposes. We only included studies with participants using the documents’ language as their first language. Finally, we excluded studies with participants suffering from language or reading disorders such as aphasia or dyslexia.
We included studies published in journals, conference proceedings or books from 1980 on. Although there is some arbitrariness to this starting moment, our impression is that most of the comprehension research relevant to text design has been done from 1980 on. The choice for this time frame excludes the classic readability studies by Flesch, Bormut, and others, but these studies concentrate on predictive validity and not on text improvement. The included studies are peer-reviewed publications in English, French, German, Spanish, or Dutch journals or books. However, the overwhelming majority of studies are in English. This is not just because our searches used English terms. In fact, English search terms do yield non-English hits, and even more so than search terms in the original language. For instance, readability, when entered in the Scopus search Engine, yields 4648 hits, of which 163 are in German and 87 in French. In contrast, the term Verständlichkeit provides only 58 German studies, and Lesbarkeit 23. Finally, review articles were not included, although they were inspected for relevant references.
We note that the knowledge base takes the study as the basic entity. This means that articles reporting two studies yield two knowledge base entries, provided that both studies meet our requirements. And this means that a study reported twice in different publications is a single entity. For instance, we included Murray (1997), but discarded Murray (1995) which reports the same study.
No matter how carefully requirements are defined, some cases will fall in a grey area. For instance, studies by McNamara and colleagues of educational text often use powerful and complex manipulations. O’Reilly and McNamara (2007) extend a text from 650 to 901 words by replacing pronouns by noun phrases, inserting connectives and signaling phrases and adding lexical material to increase conceptual overlap between sentences. Moreover, they insert topic sentences and ‘descriptive elaborations.’ These last interventions appear to alter message content, while the first group can be seen as signaling the referential and relational coherence. Because of its importance for issues of readability, this work was included in the database. We noted the complexity of the intervention in an annotation. Another example is the work on ‘seductive details’ by Schraw (1998) and others. These details add content, but content irrelevant to the main story line. Finally, the work by Sadoski and colleagues on text concreteness (for example, Sadoski et al., 2000) uses different texts instead of text versions, which reduces its relevance for document design somewhat, since differences in concreteness often reflect differences in text topic, not style. This work was also included in the database, because it compares concreteness effects with the effects of traditional text complexity features. Again, an annotation is included on the issue of the comparison of texts with varied content.
Searching Studies on Comprehension
We started using eight search engines, comparing the results for identical search term combinations and checking whether certain classical papers turn up in the output. Five engines were selected:
- Scopus provides high quality results, covers many sources, and is easy to use;
- Google Scholar gives enormous numbers of results, but enables the use of very extensive search terms combinations to zoom in on relevant work;
- PsycINFO appears to yield results that do not turn up in the other engines;
- LLBA provides high quality results in the domain of linguistics and psychological studies into language behavior;
- Web of Science is one of the dominant databases in the domain of social sciences, indexing also journals in the field of technical communication.
We started out with general search terms and combinations such as ‘text AND comprehension’. All engines returned large numbers of results, but few relevant studies. Adding terms to the search combinations improved the quality of results. In order to build these expanding search strings in a principled way, we constructed a general string syntax composed by three classes of search terms:
- Terms referring to message features (103 features were listed);
- Terms referring to comprehension effects (18 effect types were listed);
- Terms referring to text genres (for example, patient leaflets) and/or text features (word, sentence, text) (32 genres and 13 text features were listed).
Our search combinations always had two of these types of terms. Our research team used all possible combinations systematically in all five selected search engines. For instance, one of our search combinations was language comprehension (effect term) AND word familiarity (text feature), which yielded 39 results in Scopus. To manage the length of results lists, we added new terms to the combinations in case of more than 200 results. For instance, text AND comprehension gives 6.651 results in Scopus. Adding experiment reduces the number to 983; and adding the text genre news reduces the list size to 11. Examples of term combinations include:
- word difficulty AND text comprehension,
- passives AND reading comprehension,
- comprehension effect AND word repetition AND news reports,
- understanding AND legal text AND improve,
- comprehension history text AND experiment.
Apart from these searches, a number of journals and proceedings have been searched manually. More importantly, the hits have been used to produce further candidate studies by checking their references and by checking studies referencing them. Our searches often resulted in considerable numbers of hits that had to be scanned using the criteria we discussed above. For instance, one of our combinations, Readability AND “Informed consent information” yields 85 studies published after 1979 in Scopus, five of which qualify for the knowledge base.
Bjørn, E., Rossel, P., & Holm, S. (1999). Can the written information to research subjects be improved? An empirical study. Journal of Medical Ethics, 25(3), 263-267.
Kang et al. (2009). Informed consent recall and comprehension in orthodontics: Traditional vs. improved readability and processability methods. American Journal of Orthodontics and Dentofacial Orthopedics, 136(4), 488.e1-488.e13.
Knapp, P., Raynor, D. K., Silcock, J., & Parkinson, B. (2009). Performance-based readability testing of participant information for a phase 3 IVF trial. Trials 10, art. no. 1745, 79.
Paris, A., Brandt, C., Cornu, C., Maison, P., Thalamas, C., & Cracowski, J. (2010). Informed consent document improvement does not increase patients’ comprehension in biomedical research. British Journal of Clinical Pharmacology, 69(3), 231-237.
Tait, A. R., Voepel-Lewis, T., Malviya, S., & Philipson, S. J. (2005). Improving the readability and processability of a pediatric informed consent document: Effects on parents’ understanding. Archives of Pediatrics Adolescent Medicine, 159(4), 347-352.
Three examples of rejected studies are presented below.
| Barrio-Cantalejo, I. M. et al. (2009). Cross-cultural adaptation and validation of the picker patient experience questionnaire-15 for use in the Spanish population. | This study deals with adapting a patient satisfaction questionnaire. |
| Kellen, E. et al. (2010). Carefully weighing the advantages and disadvantages of the screening program for breast cancer in Flanders. | This is a medical study on the effectivity of screening programs. |
| Ménoni, V. et al. (2010). The readability of information and consent forms in clinical research in France. | This study applies a readability formula, without investigating actual comprehension. |
Annotation
Having selected the studies, we needed to annotate them. Our aim was to provide the user with multiple routes of access to a study. Besides being findable on the traditional bibliographical parameters such as author and title, we aimed at a number of menus in which users may click on features we expect to be relevant for both researchers and practitioners. For instance, a question to be answered might be: what studies have been done on the role of text structure for the comprehension of patient information leaflets? We developed the following annotation model.
Basic Information
The model starts with general bibliographic data: author, year of publication, title, and source information. The results pages also provide a Google Scholar icon, so that users authorized to access the publication are able to view it immediately. For articles reporting several studies, a study number is provided.
Abstract and Conclusion
The results page provides a copy of the abstract (in the original language). Moreover, there is a field providing important conclusions, drawn from the abstract or from the discussion in the body text of the paper.
Domain and Genre
We categorized the domain and the genres for the investigated documents. The domain is the societal realm in which the text occurs. Genres are categories of conventional documents.
Message Features and Feature Groups
In Table 1, we can see that some genre groups may show overlap. For instance, patient information about an asthma inhaler may be regarded as medicine information, but it may also be categorized as a manual for a device. In such cases, the topic dimension was chosen as the primary criterion. Similarly, mortgage conditions have been classified as financial texts and not as legal texts, and popular science texts about health topics have been annotated as health texts and not as science texts. Furthermore, not all genres are restricted to one domain. For instance, advisory reports, advertisements, forms and questionnaires can occur freely in every domain. This is why we also categorized genres without domain restrictions. Finally, we need to accommodate the fact that a considerable number of psycholinguistic studies use artificial, ‘non-genre’ texts, or ‘textoids’ as they have been called: isolated sentences or text fragments, that are purely constructed for experimental purposes and do not serve any ‘natural’ function.
The knowledge base uses thirteen feature groups, listed and illustrated in Table 2. These were the features we found in our collection of research papers. Some features may look rather technical, like type token ratio or propositional density. The type-token ratio is utilized in language studies to evaluate the lexical diversity in a document. Propositional density refers to the number of ideas in a piece of discourse relative to the number of words. Information density is often used as an indication of complexity.
Besides messages features, the studies have also been annotated in terms of well-known writing advices, in order to enhance its accessibility to practitioners. Examples of such advice are ‘explain difficult words’ and ‘use list formats for enumerations.’ The advice list is shorter than the message feature list, as some features have been studied but not yet used in advices.
Comprehension Measures and Reading Process Measures
Comprehension measures are the questions and assignments participants in the studies respond to in order to demonstrate understanding, such as multiple choice questions or cloze tests. Most of these categories were clear-cut, with the exception of ‘task performance.’ This category refers to studies measuring the accuracy of carrying out text instructions, but also to application tasks in which participants used text information in solving scenario questions.
Stimuli and Type of Manipulation
We coded the modality of the stimulus (spoken, written, and digital) and its size: complete texts may be studied, sentences (for instance in an experiment on active and passive constructions), isolated words (in a word frequency study), or separate visuals. To enable an assessment of the generalizability of the results, we annotated the number of stimuli used in the study. We also noted whether the stimuli (or examples of the stimuli) are given in the report, which is unfortunately not always the case.
Finally, intervention complexity is coded. In case of complex interventions, several message features are varied simultaneously, which hinders causal explanations for the results. Complex intervention studies may be very important from a practical point of view and may encourage further research, but they are obviously limited in terms of both internal validity and generalizability.
Participant Information
The age, educational level, and number of participants are noted. This information is useful to enable prima facie assessments on generalizability.
Control Variables and Non-Textual Independent Variables
A study’s internal validity can be considerably enhanced when potential confounding variables are controlled for, such as participants’ abilities or reading task features. Hence we note such control or moderator variables. When textual variables have been crossed with non-textual independent variables, the robustness of the message feature effects can be better assessed, as well as the potential interactions between message effects and participant or task characteristics.
Results: An Overview of the Studies in the Knowledge Base
One of the interesting results of all these searches and annotations is that it uncovers the main interests of researchers into comprehension studies. What genres do they focus on? And which text features are studied most intensively? Currently, the knowledge base contains 703 studies, drawn from 479 publications. Best represented are the genre groups science (168), health (132) and education (120). Science texts or popular science texts are often used in experiments with students as participants. The educational texts are usually taken from textbooks used in secondary education. The most common health text genre is medicine information, but in this domain we also find informed consent documents and health warnings. Strikingly, the financial and legal domains are poorly represented with six and fifteen comprehension studies respectively.
We can also see which features of communication products are most popular in comprehension research. Table 3 presents a list of the message feature groups and the number of studies devoted to different features.
Within the visual group, illustrations are the most frequently researched genre with 97 studies. Other visual genres in the knowledge base are diagrams (48), animations (36), icons and pictograms (26), graphs (21) and tables (21). Many of the illustration studies are conducted in the educational domain and some of them in the health domain and the domain of operating devices.
Of the textual feature groups, most attention goes to macro-level signals, especially headings and organizers, and to text structure (especially the order of text components).
As for the comprehension measures, multiple-choice and open comprehension questions are most frequently used. Other measures are free recall, recognition, task performance and cued recall. Summarizing and searching tasks are less popular, and the same applies to cloze tests (Table 4).
All this concerns measures of comprehension products, that is, mental representations of text information. The knowledge base also provides online (process) measures, if applicable. The most popular processing measures are reading time (219 studies) and response time (135). Another option is analyzing the reading path through the text (38). Sometimes this concerns eye movement measures such as the number of regressions to earlier material, but more often macro-level path phenomena are at issue, such as the navigation trail through a Web site or switching between text and illustrations.
Finally, the knowledge base informs us on another factor that crucially affects comprehension: the readers that have to do the work. Table 5 shows that in 190 studies reader variables have been taken into account; prior knowledge seems to be the most interesting factor.
Case Study: Comprehending Active and Passive Sentences
Let us now probe a little further and try to answer the scenario question of our persona Kate Patel, who had a debate with her contractor about passive voice in manuals. What evidence may Kate find in the Knowledge Base sustaining her claim that the passive voice may be used?
Searching on this message feature (which appears when the group of the ‘sentence level features’ is selected), we find 15 studies published after 1980. Most studies presented isolated sentences to participants. Kate understands that no manual fragments have been used in these studies. Of the more recent studies, five studies use a complex intervention: Handel et al. (2001), Kang et al. (2009), Leroy et al. (2010), Ulijn and Strother (1990), and Wenger and Spyridakis (1993). For instance, Ulijn and Strother (1990) simplified the construction of 10 sentences; in two of them they manipulated passives along with other features, and in one of them they only manipulated the passive. In their comprehension data, all sentences are aggregated. And Kang et al. (2009) combined the passive manipulation with lexical and visual changes. These studies are hard to interpret in terms of effects of passive voice. The remaining studies are surveyed in Table 6.
Bostian (1983) and Lenzner et al. (2010) are the only studies to date that investigate passive manipulations in authentic texts. Unfortunately, Bostian’s study is not well-designed, as there is an alternative explanation for the slower reading of passive sentences: the author mentions that they were longer than the active sentences (15.7 versus 13.5 words). The interesting feature of the Lenzner et al. study is its process measure: the time needed to answer the question. The passive manipulation in his study only concerned two questions.
The rest of the studies are psycholinguistic experiments using isolated sentences as stimuli. Carrithers (1989) uses reading time as comprehension measure. Although it is assumed that passive sentences are more complex than their active counterparts, his experiment does not show differences in reading times, except when the passive sentences contain ergative verbs such as amaze. The work of Ferreira, Street and Dabrowska uses direct comprehension measures such as question answering or picture matching. They find that passive sentences are more often misunderstood, but this applies specifically to less-educated participants reading implausible sentences. Street and Dabrowska (2010) further show that a training in reading passive sentences for those participants is fairly effective, which indicates that the problem has to do with a lack of experience with the passive construction and not with working memory.
Our persona Kate may conclude that the only empirical evidence for the difficulty of passive voice is related to a less-educated audience. If her manual is produced for higher educated readers, their comprehension may not suffer from passive sentences. She could have found this information in a university library spending some hours using different search engines, thinking about combinations of terms and scanning hundreds of titles leading to these papers. But using the knowledge base at home, she can learn about the results of these studies without consulting the journals and without having to read complicated methods and results sections.
As a professional writer, she will not change her writing strategy to describe actions in the active voice. She will present specific information—like negative consequences of actions for the reader—in passive voice, as she has been doing for years. Surprisingly, the knowledge base does not show empirical evidence for such expert knowledge about the effective use of passive voice, like presented on the Web site www.grammar-monster.com. In that way, the knowledge base is useful for both practitioners and researchers: it helps to find empirical evidence on issues concerning comprehension and it sheds light on issues that need more or new research.
Further Work
This knowledge base is the first digital database specifically dedicated to research on comprehensible language and text design. The first release of such a resource certainly has its limitations. For one thing, our impression is that book publications are underrepresented, as many search engines seem to concentrate on journal publications. And in the short time frame for the project, we cannot even be sure to have covered all relevant journal publications. For instance, we suspect that there are much more studies on text presentation (typography and lay-out) than we have located so far. Hence we invite all users to send in suggestions about studies that we have missed.
The current version of the knowledge base contains studies up to 2010. The database will be updated in 2014. We hope to secure further funding for following updates. More importantly, we would welcome international cooperation on the further development of the resource, both in terms of its content and on
its usability. We certainly do not pretend that our research base is exhaustive for the technical communication body of knowledge. The Technical Communication Body of Knowledge (TCBOK) demonstrates how broad a knowledge base for the profession can be, ranging from academic programs to marketing tools and career paths. However, we do believe the Knowledge Base Comprehensible Text presents a serious attempt to outline the empirical research on a crucial aspect of the profession: producing comprehensible information.
Acknowledgment
The Knowledge Base Comprehensible Text was supported by the Dutch National Science Foundation NWO (Comprehensible Language and Effective Communication, project. nr. 321-70-050).
References
Bostian, L. R. (1983). How active, passive and nominal styles affect readability of science writing. Journalism Quarterly, 60, 635-40, 670.
Carrithers, C. (1989). Syntactic complexity does not necessarily make sentences harder to understand. Journal of Psycholinguistic Research, 18(1), 75-88.
Coppola, N. W. (2010). The technical communication body of knowledge initiative: An academic-practitioner partnership. Technical Communication, 57, 11-25.
Dabrowska, E., & Street, J. (2006). Individual differences in language attainment: Comprehension of passive sentences by native and non-native English speakers. Language Sciences, 28, 604-615.
Ferreira, F. (2003). The misinterpretation of noncanonical sentences. Cognitive Psychology, 47, 164-203.
Handel, D., Fluck, H. -., Forster, M., & Blaha, M. (2001). Bürgernahe Verwaltungstexte: Ein Kooperationsprojekt der Stadt Bochum und der Ruhr-Universität zur Textoptimierung. Fachsprache, 23(3-4), 139-152.
Kang, E. Y., Fields, H. W., Kiyak, A., Beck, F. M., & Firestone, A. R. (2009). Informed consent recall and comprehension in orthodontics: Traditional vs. improved readability and processability methods. American Journal of Orthodontics and Dentofacial Orthopedics, 136(4), 488.e1-488.e13.
Lenzner, T., Kaczmirek, L., Lenzner, A. (2010). Cognitive burden of survey questions and response times: A psycholinguistic experiment. Applied Cognitive Psychology, 24(7), 1003-1020.
Leroy, G., Helmreich, S., & Cowie, J. R. (2010). The influence of text characteristics on perceived and actual difficulty of health information. International Journal of Medical Informatics, 79(6), 438-449.
Meyer, B. J., Marsiske, M., & Willis, S. L. (1993). Text processing variables predict the readability of everyday documents read by older adults. Reading Research Quarterly, 28(3), 234-249.
Murray, J. D. (1995). Logical connectives and local coherence. In R. F. J. Lorch & E. J. O’Brien (Eds.), Sources of coherence in reading (pp. 107-125). Hillsdale, NJ, England: Lawrence Erlbaum Associates.
Murray, J. D. (1997). Connectives and narrative text: The role of continuity. Memory & Cognition, 25(2), 227-236.
Murray, N. M., Manrai, L. A., & Manrai, A. K. (1993). Public policy relating to consumer comprehension of television commercials: A review and some empirical results. Journal of Consumer Policy, 16(2), 145-170.
O’Reilly, T., & McNamara, D. S. (2007). Reversing the reverse cohesion effect: Good texts can be better for strategic, high-knowledge readers. Discourse Processes, 43(2), 121-152.
Palmer, J. S. & Killingsworth, M. J. (2002). Research and consulting in technical communication. Technical Communication Quarterly, 11, 389-409.
Sadoski, M., Goetz, E. T., & Rodriguez, M. (2000). Engaging texts: Effects of concreteness on comprehensibility, interest, and recall in four text types. Journal of Educational Psychology, 92(1), 85-95.
Salmeron, L., Kintsch, W., & Kintsch, E. (2010). Self-regulation and link selection strategies in hypertext. Discourse Processes, 47(3), 175-211.
Schraw, G. (1998). Processing and recall differences among seductive details. Journal of Educational Psychology, 90(1), 3-12.
Street, J. A., & Dabrowska, E. (2010). More individual differences in language attainment: How much do adult native speakers of English know about passives and quantifiers? Lingua, 120(8), 2080-2094.
Ulijn, J. M., & Strother, J. B. (1990). The effect of syntactic simplification on reading EST texts as L1 and L2. Journal of Research in Reading, 13(1), 38-54.
Wenger, M. J., & Spyridakis, J. H. (1993). Reduced text structure at two text levels: Impacts on the performance of technical readers. Journal of Technical Writing and Communication, 23, 333-352.
About the Authors
Leo R. Lentz is professor of document design and communication at Utrecht University. Web usability and text evaluation is the main focus of his research. He develops evaluation methods for reader-focused and text-focused evaluation. Contact: l.r.lentz@uu.nl.
Henk L.W. Pander Maat is an associate professor at the Utrecht Institute of Linguistics OTS at Utrecht University. His main research interests are readability and usability research, document design, and genre analysis. Contact: h.l.w.pandermaat@uu.nl.
Ted J.M.Sanders is professor of discourse studies and language use at Utrecht University. His research concentrates on discourse structure and coherence.Contact: t.j.m.sanders@uu.nl
Manuscript received 17 December 2012; revised 17 January 2014; accepted 20 January 2014.
