The DOI matches its Crossref record; 114 of the 115 references it cites were found in a registry or an index.
Computational Analyses of Multilevel Discourse Comprehension
Open accessAbstract
The proposed multilevel framework of discourse comprehension includes the surface code, the textbase, the situation model, the genre and rhetorical structure, and the pragmatic communication level. We describe these five levels when comprehension succeeds and also when there are communication misalignments and comprehension breakdowns. A computer tool has been developed, called Coh-Metrix, that scales discourse (oral or print) on dozens of measures associated with the first four discourse levels. The measurement of these levels with an automated tool helps researchers track and better understand multilevel discourse comprehension. Two sets of analyses illustrate the utility of Coh-Metrix in discourse theory and educational practice. First, Coh-Metrix was used to measure the cohesion of the text base and situation model, as well as potential extraneous variables, in a sample of published studies that manipulated text cohesion. This analysis helped us better understand what was precisely manipulated in these studies and the implications for discourse comprehension mechanisms. Second, Coh-Metrix analyses are reported for samples of narrative and science texts in order to advance the argument that traditional text difficulty measures are limited because they fail to accommodate most of the levels of the multilevel discourse comprehension framework.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Kilgarriff, A. (2000) · Language · doi:10.2307/417141 · Cited by 11,529
Bruner, J. (1986) · doi:10.4159/9780674029019 · Cited by 8,657
Eye movements in reading and information processing: 20 years of research
Rayner, K. (1998) · Psychological Bulletin · doi:10.1037/0033-2909.124.3.372 · Cited by 7,979
Building a Large Annotated Corpus of English: The Penn Treebank
Marcus, M. (1993) · doi:10.21236/ada273556 · Cited by 7,405
Halliday, M. A. K., & Hasan, R. (2014) · Routledge eBooks · doi:10.4324/9781315836010 · Cited by 7,392
Actual Minds, Possible Worlds.
Mergler, N. L., Schleifer, R., & Bruner, J. S. (1986) · MLN · doi:10.2307/2905728 · Cited by 6,626
Landauer, T. K., & Dumais, S. T. (1997) · Psychological Review · doi:10.1037/0033-295x.104.2.211 · Cited by 6,282
Strategies of Discourse Comprehension
Staczek, J. J., van Dijk, T. A., & Kintsch, W. (1985) · Modern Language Journal · doi:10.2307/326517 · Cited by 5,377
Toward a model of text comprehension and production.
Kintsch, W., & van Dijk, T. A. (1978) · Psychological Review · doi:10.1037/0033-295x.85.5.363 · Cited by 5,298
Variation across Speech and Writing
Biber, D. (1988) · doi:10.1017/cbo9780511621024 · Cited by 5,227
Research works printed but not linked: 16
- Baayen R. H., 1995, The CELEX Lexical Database (Release 2) {CD‐ROM}
- Charniak E., 2000, Proceedings of the First Conference on North American Chapter of the Association for Computational Linguistics, 132
- Gough P. B., 1972, Language by ear and by eye, 331
- Hempelmann C. F., 2005, Proceedings of the 27th Annual Meetings of the Cognitive Science Society, 941
- Hewitt P. G., 1998, Conceptual physics
- Koslin B. I., 1987, The DRP: An effective measure in reading
- McCarthy P. M., 2006, Proceedings of the Florida Artificial Intelligence Research Society International Conference (FLAIRS), 764
- McClelland J. L., 1986, Parallel distributed processing, 122
- McNamara D. S., Assessing reading in the 21st century: Aligning and applying advances in the reading and measurement sciences
- McNamara D. S., 2008, Towards a comprehensive model of comprehension, The psychology of learning and motivation, 297
- Meyer B. J. F., 2007, Reading comprehension strategies: Theories, interventions, and technologies, 347
- Pennebaker J. W., 2007, Linguistic inquiry and word count
- Prince E. F., 1981, Radical pragmatics, 223
- Rayner K., 1994, The psychology of reading
- VanderVeen A., 2007, Theories of text comprehension: The importance of reading strategies to theoretical foundations of reading comprehension, 137
- Williams P. J., 2007, Reading comprehension strategies: theories, interventions, and technologies, 199
Cited by
Sorted by citations
Research Methods in Applied Linguistics
(2021) · Research methods in applied linguistics · doi:10.1075/rmal · Cited by 2,782
Automated Evaluation of Text and Discourse with Coh-Metrix
McNamara, D. S., Graesser, A. C., McCarthy, P. M., & Cai, Z. (2014) · Cambridge University Press eBooks · doi:10.1017/cbo9780511894664 · Cited by 932
Self-citationDelgado, P., Vargas, C., Ackerman, R., & Salmerón, L. (2018) · Educational Research Review · doi:10.1016/j.edurev.2018.09.003 · Cited by 664
Graesser, A. C., McNamara, D. S., & Kulikowich, J. M. (2011) · Educational Researcher · doi:10.3102/0013189x11413260 · Cited by 635
“The five levels are elaborated later and in Graesser and McNamara (2011).”
BackgroundMethodInfluentialSelf-citationReading from paper compared to screens: A systematic review and meta‐analysis
Clinton‐Lisell, V. (2019) · Journal of Research in Reading · doi:10.1111/1467-9817.12269 · Cited by 588
Goldman, S. R., Britt, M. A., Brown, W. R., Cribb, G., George, M., Greenleaf, C. L., Lee, C. D., Shanahan, C. R., & Project READI (2016) · Educational Psychologist · doi:10.1080/00461520.2016.1168741 · Cited by 396
“…that originated in the work of Kintsch and colleagues (Kintsch, 1998; van Dijk & Kintsch, 1983) and that have been elaborated by a variety of discourse researchers (e.g., Goldman, 2004; Goldman, Varma, & Cot e, 1996; Graesser & McNamara, 2010; van den Broek, Young, Tzeng, & Linderholm, 1999).”
Background21st-Century Readers: Developing Literacy Skills in a Digital World
OECD (2021) · PISA · doi:10.1787/a83d84cb-en · Cited by 337
A hierarchical classification approach to automated essay scoring
McNamara, D. S., Crossley, S. A., Roscoe, R. D., Allen, L. K., & Dai, J. (2014) · Assessing Writing · doi:10.1016/j.asw.2014.09.002 · Cited by 243
Self-citationEason, S. H., Goldberg, L. F., Young, K. M., Geist, M. C., & Cutting, L. E. (2012) · Journal of Educational Psychology · doi:10.1037/a0027182 · Cited by 229
“Studies showing that inferences are generated quickly and reliably by all readers for narrative texts (see Graesser & McNamara, 2011; Haberlandt & Graesser, 1985) may explain why inferences and reasoning did not make significant contributions to the Narrative passages in our study.”
BackgroundMind wandering while reading easy and difficult texts
Feng, S., D’Mello, S., & Graesser, A. C. (2013) · Psychonomic Bulletin & Review · doi:10.3758/s13423-012-0367-y · Cited by 257
Self-citationNatural language processing in an intelligent writing strategy tutoring system
McNamara, D. S., Crossley, S. A., & Roscoe, R. D. (2012) · Behavior Research Methods · doi:10.3758/s13428-012-0258-1 · Cited by 232
Self-citationCoh-Metrix Measures Text Characteristics at Multiple Levels of Language and Discourse
Graesser, A. C., McNamara, D. S., Cai, Z., Conley, M. W., Li, H., & Pennebaker, J. W. (2014) · The Elementary School Journal · doi:10.1086/678293 · Cited by 199
“…movefromwordsinto sentence interpretation, construction of inferences, use of background knowledge, reasoning, and knowledge of discourse structures (Graesser & McNamara, 2011 ; Graesser, Singer, & Trabasso, 1994 ; Kintsch, 1998 ; McNamara, 2007 ; Perfetti, 1999 ; Snow, 2002 ).”
BackgroundSelf-citationGRAESSER, A., & McNAMARA, D. (2010) · Educational Psychologist · doi:10.1080/00461520.2010.515933 · Cited by 153
Self-citationDelgado, P., & Salmerón, L. (2020) · Learning and Instruction · doi:10.1016/j.learninstruc.2020.101396 · Cited by 188
Perea, M., Panadero, V., Moret‐Tatay, C., & Gómez, P. (2012) · Learning and Instruction · doi:10.1016/j.learninstruc.2012.04.001 · Cited by 136
McNamara, D. S., Ozuru, Y., & Floyd, R. G. (2011) · lnternational Electronic Journal of Elementary Education · Cited by 132
Self-citationThe Writing Pal Intelligent Tutoring System: Usability Testing and Development
Roscoe, R. D., Allen, L. K., Weston, J. L., Crossley, S. A., & McNamara, D. S. (2014) · Computers & composition/Computers and composition · doi:10.1016/j.compcom.2014.09.002 · Cited by 123
“…developed over several years of iterative, interdisciplinary work that synthesizes best practices from writing pedagogy and user-centered design (Dai, Raine, Roscoe, Cai, & McNamara, 2011; McNamara et al., 2012), with contributions from psychology, composition, linguistics, and computer science.”
BackgroundMethodInfluentialSelf-citationEngaging Struggling Adolescent Readers to Improve Reading Skills
Kim, J. S., Hemphill, L., Troyer, M., Thomson, J., Jones, S. M., LaRusso, M. D., & Donovan, S. (2016) · Reading Research Quarterly · doi:10.1002/rrq.171 · Cited by 139
“As readers produce a situation model (Kintsch, 1998; Zwaan & Radvansky, 1998), they integrate background knowledge with the literal textbase (Graesser & McNamara, 2011; Zwaan, 1994).”
BackgroundMcNamara, D. S., & Graesser, A. C. (2012) · IGI Global eBooks · doi:10.4018/978-1-60960-741-8.ch011 · Cited by 105
Self-citationConnectives as Processing Signals: How Students Benefit in Processing Narrative and Expository Texts
van Silfhout, G., Evers-Vermeul, J., & Sanders, T. J. M. (2014) · Discourse Processes · doi:10.1080/0163853x.2014.905237 · Cited by 99
“As a result, students spend a lot of cognitive effort on generating inferences to fill in or explain coherence gaps (Graesser & McNamara, 2011).”
Foster, M. I., & Keane, M. T. (2015) · Cognitive Psychology · doi:10.1016/j.cogpsych.2015.08.004 · Cited by 103
“So, the comprehension of a scenario is assumed to involve building a situation model in working memory (e.g., Graesser & McNamara, 2011; O’Brien, Rizzella, Albrecht & Halleran, 1998; Zwaan, Magliano, & Graesser, 1995; Zwaan & Radvansky, 1998).”
Interest-based text preference moderates the effect of text difficulty on engagement and learning
Fulmer, S. M., D’Mello, S. K., Strain, A. C., & Graesser, A. C. (2014) · Contemporary Educational Psychology · doi:10.1016/j.cedpsych.2014.12.005 · Cited by 114
Self-citationFrancis, D. J., Kulesz, P. A., & Benoit, J. S. (2018) · Remedial and Special Education · doi:10.1177/0741932518772904 · Cited by 98
“…cohesion (i.e., low overlap of words and concepts across the text), low deep cohesion (i.e., less connecting words used to clarify relations between information in the text), and expository genre characterizing more difficult texts (Graesser & McNamara, 2011; McNamara, 2001; McNamara et al., 1996).”
BackgroundParticipant, rater, and computer measures of coherence in posttraumatic stress disorder.
Rubin, D. C., Deffler, S. A., Ogle, C. M., Dowell, N. M., Graesser, A. C., & Beckham, J. C. (2016) · Journal of Abnormal Psychology · doi:10.1037/abn0000126 · Cited by 88
“…le is in te nd ed so le ly fo r th e pe rs on al us e of th e in di vi du al us er an d is no t to be di ss em in at ed br oa dl y. and discourse (Graesser & McNamara, 2011; Kintsch, 1998; Snow, 2002), which distinguish representations of meaning, structures, strategies, and cognitive processes…”
Self-citationCOVID-19 Pandemic and Student Reading Achievement: Findings From a School Panel Study
Ludewig, U., Kleinkorres, R., Schaufelberger, R., Schlitter, T., Lorenz, R., König, C., Frey, A., & McElvany, N. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.876485 · Cited by 112
“Reading achievement involves multiple levels of text comprehension: surface structure, text base, situation model, rhetorical structure, and pragmatic communication (Kintsch, 1988; Graesser and McNamara, 2011).”
BackgroundOn Thinking and STEM Education
Li, Y., Schoenfeld, A. H., diSessa, A. A., Graesser, A. C., Benson, L., English, L. D., & Duschl, R. A. (2019) · Journal for STEM Education Research · doi:10.1007/s41979-019-00014-x · Cited by 92
“Furthermore, thinking can be differentiated into levels, similar to how diSessa (2015) defends studying knowledge according to different levels and Graesser and McNamara (2011) differentiate discourse comprehension into multiple levels.”
BackgroundSelf-citationComputational Methods to Extract Meaning From Text and Advance Theories of Human Cognition
McNamara, D. S. (2010) · Topics in Cognitive Science · doi:10.1111/j.1756-8765.2010.01117.x · Cited by 77
Self-citationMills, C., Fridman, I., Soussou, W., Waghray, D., Olney, A. M., & D’Mello, S. K. (2017) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/3027385.3027431 · Cited by 76
Sung, Y., Chen, J.-L., Cha, J.-H., Tseng, H.-C., Chang, T.-H., & Chang, K.-E. (2014) · Behavior Research Methods · doi:10.3758/s13428-014-0459-x · Cited by 70
“The results of our study provide further evidence supporting the claim that the multilevel approach in readability analysis better represents the complexities of texts and the comprehension process (Graesser & McNamara, 2011; Graesser et al., 2004; McNamara, Louwerse, McCarthy, & Graesser, 2010).”
ResultCrossley, S. A., & McNamara, D. S. (2011) · International Journal of Continuing Engineering Education and Life-Long Learning · doi:10.1504/ijceell.2011.040197 · Cited by 103
Self-citationMiddle School Reading Assessment: Measuring What Matters Under a RTI Framework
O’Reilly, T., Sabatini, J., Bruce, K., Pillarisetti, S., & McCormick, C. M. (2012) · Reading Psychology · doi:10.1080/02702711.2012.631865 · Cited by 67
“Since many important text features interact significantly with genre (see Graesser & McNamara, 2011 ; Sheehan, Kostin, & Futagi, 2008a; 2008b ), distinct text difficulty models have been developed for three targeted genres so far: literary, informational, and persuasive.”
BackgroundExploring development of social capital in a CMOOC through language and discourse
Joksimovíc, S., Dowell, N., Poquet, O., Kovanović, V., Gašević, D., Dawson, S., & Graesser, A. C. (2017) · The Internet and Higher Education · doi:10.1016/j.iheduc.2017.09.004 · Cited by 69
“This was achieved in a study that examined 53 Coh-Metrix measures for 37,520 texts in the TASA (Touchstone Applied Science Association) corpus, which represents what typical high school students have read throughout their lifetime (Graesser et al., 2011).”
BackgroundMethodInfluentialSelf-citationModeling Learners' Social Centrality and Performance through Language and Discourse.
Dowell, N., Poquet, O., Joksimovíc, S., Graesser, A. C., Dawson, S., Gašević, D., Hennis, T., de Vries, P., & Kovanović, V. (2015) · Edinburgh Research Explorer (University of Edinburgh) · Cited by 65
Self-citationBarth, A. E., Tolar, T. D., Fletcher, J. M., & Francis, D. J. (2013) · Journal of Educational Psychology · doi:10.1037/a0033826 · Cited by 63
“Graesser and McNamara (2011) reported that 54% of the variance in word reading times was accounted by four groups of variables that included material layout, surface code, textbase/situation model, and genre.”
BackgroundMethodResultInfluentialComputer-based assessment of student-constructed responses
Magliano, J. P., & Graesser, A. C. (2012) · Behavior Research Methods · doi:10.3758/s13428-012-0211-3 · Cited by 61
“For example, Coh-Metrix (Graesser & McNamara, 2011; Graesser, McNamara, & Kulikowich, 2011; McNamara, Louwerse, McCarthy, & Graesser, 2010) automatically analyzes texts on a wide range of linguistic and discourse features that correspond to genre, referential cohesion, situation model cohesion, syntax, and word concre…”
BackgroundMethodInfluentialSelf-citationNgao, A. I., Sang, G., & Kihwele, J. E. (2022) · Education Sciences · doi:10.3390/educsci12080549 · Cited by 60
The face of contagion: consumer response to service failure depiction in online reviews
Fox, A. K., Deitz, G. D., Stafford, M. R., & Fox, J. D. (2018) · European Journal of Marketing · doi:10.1108/ejm-12-2016-0887 · Cited by 59
Schwabe, A., Lind, F., Kosch, L., & Boomgaarden, H. G. (2022) · Media Psychology · doi:10.1080/15213269.2022.2070216 · Cited by 75
“On the textual level, they consist of more connectives and present more temporal cohesion, both of which benefit reading comprehension (Graesser & McNamara, 2011).”
BackgroundEcker, U. K. H., Butler, L. H., & Hamby, A. M. (2020) · Cognitive Research Principles and Implications · doi:10.1186/s41235-020-00266-x · Cited by 60
Important text characteristics for early-grades text complexity.
Fitzgerald, J., Elmore, J., Koons, H. H., Hiebert, E. H., Bowen, K., Sanford, E. E., & Stenner, A. J. (2014) · Journal of Educational Psychology · doi:10.1037/a0037289 · Cited by 56
Effects of instructional conditions and experience on the adoption of a learning tool
Gašević, D., Mirriahi, N., Dawson, S., & Joksimovíc, S. (2016) · Computers in Human Behavior · doi:10.1016/j.chb.2016.10.026 · Cited by 60
Kulesz, P. A., Francis, D. J., Barnes, M. A., & Fletcher, J. M. (2016) · Journal of Educational Psychology · doi:10.1037/edu0000126 · Cited by 60
Sourcing in professional education: Do text factors make any difference?
Bråten, I., Strømsø, H. I., & Andreassen, R. (2016) · Reading and Writing · doi:10.1007/s11145-015-9611-y · Cited by 50
Investigating the affordances of OpenAI's large language model in developing listening assessments
Aryadoust, V., Zakaria, A., & Jia, Y. (2024) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2024.100204 · Cited by 67
van Silfhout, G., Evers-Vermeul, J., Mak, W. M., & Sanders, T. J. M. (2014) · Journal of Educational Psychology · doi:10.1037/a0036293 · Cited by 47
“In other words, connectives help readers to identify text structure and mentally connect ideas in a text (Degand & Sanders, 2002; Graesser & McNamara, 2011; Sanders & Noordman, 2000; Sanders & Spooren, 2007).”
BackgroundMultiple Agent Designs in Conversational Intelligent Tutoring Systems
Lippert, A., Shubeck, K. T., Morgan, B., Hampton, A. J., & Graesser, A. C. (2019) · Technology Knowledge and Learning · doi:10.1007/s10758-019-09431-8 · Cited by 58
“Each lesson in CSAL AutoTutor includes one or more of the following theoretical levels of comprehension as defined by Graesser and McNamara (2011): words, syntax, textbase, situation model, and rhetorical structure.”
BackgroundSelf-citationAryadoust, V. (2014) · Educational Psychology · doi:10.1080/01443410.2014.950946 · Cited by 45
“Texts rich in one or more of these aspects are more coherent, but the presence of many causal verbs and relatively few particles can jeopardise text cohesion (Graesser & McNamara, 2011).”
BackgroundResultInfluentialIndices of cognitive effort in machine translation post-editing
Vieira, L. N. (2014) · Machine Translation · doi:10.1007/s10590-014-9156-x · Cited by 46
“0 of the Coh-Metrix automatic text analysis tool(22) (Graesser et al. 2004; Graesser and McNamara 2011) are also tested – these indices correspond here to the MT output.”
BackgroundMethodInfluentialBråten, I., Braasch, J. L. G., Strømsø, H. I., & Ferguson, L. E. (2014) · Reading Psychology · doi:10.1080/02702711.2013.864362 · Cited by 44
“It also accords with theoretical frameworks describing various levels of representation and contextual factors in discourse comprehension (Graesser & McNamara, 2011; van Dijk, 1999).”
BackgroundWalkington, C., Clinton‐Lisell, V., & Sparks, A. (2019) · Instructional Science · doi:10.1007/s11251-019-09481-6 · Cited by 45
The Language of Autocrats: Leaders' Language in Natural Disaster Crises
Windsor, L., Dowell, N., & Graesser, A. C. (2014) · Risk Hazards & Crisis in Public Policy · doi:10.1002/rhc3.12068 · Cited by 43
“…has demonstrated that translated speeches have a high correlation with the original text using Google translate, and as such we feel confident in our use of English language documents (Li, Graesser, & Cai, 2014; Shala, Rus, & Grasser, 2010; Mao et al., 1990, 1975; Graesser & McNamara 2011).”
MethodSelf-citationJoksimovíc, S., Dowell, N., Poquet, O., Kovanović, V., Gašević, D., Dawson, S., & Graesser, A. C. (2015) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/2723576.2723604 · Cited by 42
“Given that referential cohesion captures the extent to which ideas in the text are connected with each other as the text unfolds, higher referential cohesion indicates fewer gaps in conveying the ideas and increased text readability and comprehension [19].”
BackgroundSelf-citationDeeper Learning With Advances in Discourse Science and Technology
Graesser, A. C. (2015) · Policy Insights from the Behavioral and Brain Sciences · doi:10.1177/2372732215600888 · Cited by 42
“Graesser and McNamara (2011) articulated a multilevel theoretical framework that integrates the large body of research on reading comprehension in various fields, particularly at deeper comprehension levels.”
BackgroundSelf-citationMcNamara, D. S. (2013) · Discourse Studies · doi:10.1177/1461445613501446 · Cited by 41
“Cohesive cues are grounded in explicit linguistic elements (i.e. words, features, cues, signals, constituents) and their combinations (Graesser and McNamara, 2011).”
BackgroundMethodInfluentialSelf-citationWriting flexibility in argumentative essays: a multidimensional analysis
Allen, L. K., Likens, A. D., & McNamara, D. S. (2018) · Reading and Writing · doi:10.1007/s11145-018-9921-y · Cited by 40
Self-citationGoldman, S. R., Greenleaf, C. L., Yukhymenko–Lescroart, M. A., Brown, W., Ko, M. M., Emig, J. M., George, M., Wallace, P. S., Blaum, D., & Britt, M. A. (2019) · American Educational Research Journal · doi:10.3102/0002831219831041 · Cited by 41
“…the surface input, the presented information, and inferences that integrate information within and across texts and with prior knowledge (e.g., Goldman, 2004; Graesser & McNamara, 2010; Kintsch, 1994; Rouet, 2006; Rouet & Britt, 2011; Tabak, 2016; van den Broek, Young, Tzeng, & Linderholm, 1999).”
BackgroundWalkington, C., Clinton‐Lisell, V., RITTER, S., & Nathan, M. J. (2015) · Journal of Educational Psychology · doi:10.1037/edu0000036 · Cited by 42
“This finding is likely because these differences make forging connections among all of the sentences in a story problem difficult (Graesser & McNamara, 2011) and may inhibit the construction of the textbase and the situation model (McNamara et al., 2010).”
BackgroundInfluentialInvestigating boredom and engagement during writing using multiple sources of information
Allen, L. K., Mills, C., Jacovina, M. E., Crossley, S. A., D’Mello, S. K., & McNamara, D. S. (2016) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/2883851.2883939 · Cited by 39
“These components provide measures of the principal sources of text difficulty and are well aligned with an existing multilevel framework [51].”
BackgroundSelf-citationLanguage effects in international testing: the case of PISA 2006 science items
Masri, Y. E., Baird, J., & Graesser, A. C. (2016) · Assessment in Education Principles Policy and Practice · doi:10.1080/0969594x.2016.1218323 · Cited by 39
“Sentence length has long been known to be a major component of textgrade levels in readability measures such as Flesch–Kincaid grade level (Klare, 1974), presumably because it places a greater burden on working memory (Graesser & McNamara, 2011).”
BackgroundSelf-citationNILC-Metrix: assessing the complexity of written and spoken language in Brazilian Portuguese
Leal, S. E., Duran, M. S., Scarton, C., Hartmann, N. S., & Aluísio, S. M. (2023) · Language Resources and Evaluation · doi:10.1007/s10579-023-09693-w · Cited by 56
“A set of 79 metrics from NILC-Metrix was selected, based on the study by (Graesser and McNamara, 2011).”
MethodShared features of L2 writing: Intergroup homogeneity and text classification
Crossley, S. A., & McNamara, D. S. (2011) · Journal of Second Language Writing · doi:10.1016/j.jslw.2011.05.007 · Cited by 44
Self-citationFesel, S. S., Segers, E., Clariana, R. B., & Verhoeven, L. T. W. (2015) · Computers in Human Behavior · doi:10.1016/j.chb.2015.01.014 · Cited by 35
Novel Approaches to Examine Passage, Student, and Question Effects on Reading Comprehension
Miller, A. C., Davis, N. J., Gilbert, J. K., Cho, S., Toste, J. R., Street, J. A., & Cutting, L. E. (2014) · Learning Disabilities Research and Practice · doi:10.1111/ldrp.12027 · Cited by 35
“…to analyze and measure text characteristics, including surface-level (e.g., part of speech and tense) and higher-level characteristics (e.g., inferences, cohesiveness, narrativity) that are not indexed by standard text difficulty metrics, such as Flesch-Kincaid (see Graesser & McNamara, 2011).”
BackgroundVarner, L. K., Roscoe, R. D., & McNamara, D. S. (2013) · Journal of Writing Research · doi:10.17239/jowr-2013.05.01.2 · Cited by 33
“Coh-Metrix offers a broad analysis of texts, including indices of cohesion, and text difficulty at the lexical, syntactic, structural, and global levels of text (Graesser & McNamara, 2011).”
BackgroundSelf-citationWorking Memory Capacity but Not Prior Knowledge Impact on Readers' Attention and Text Comprehension
Schurer, T., Opitz, B., & Schubert, T. (2020) · Frontiers in Education · doi:10.3389/feduc.2020.00026 · Cited by 48
“This is consistent with previous studies, which showed that increasing the cohesion of a text helps to construct a situation model of the text and therefore facilitates and improves text comprehension for many readers (McNamara et al., 1996; Graesser and McNamara, 2011).”
BackgroundResultWhat Works: Creating Adaptive and Intelligent Systems for Collaborative Learning Support
Dowell, N., Cade, W., Tausczik, Y., Pennebaker, J. W., & Graesser, A. C. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-07221-0_15 · Cited by 32
“Importantly, the components aligned with the language-discourse levels previously proposed in multilevel theoretical frameworks of cognition and comprehension [7], [25,26].”
BackgroundResultInfluentialSelf-citationLexical Richness in EFL Undergraduate Students’ Academic Writing
Ha, H. S. (2019) · English Teaching · doi:10.15858/engtea.74.3.201909.3 · Cited by 32
“In order to evaluate writing performance, this study uses the text analysis tool, Coh- Matrix (Graesser & McNamara, 2011; Kim & Jeon, 2016; Lee, 2018), RANGE (Coxhead, 2000; Kim, 2014; Nation & Meara, 2010) and holistic writing scoring which reflect the overall coherence of vocabulary knowledge based on the previous r…”
MethodResearch Methods in Vocabulary Studies
Durrant, P., Siyanova‐Chanturia, A., Kremmel, B., & Sonbul, S. (2022) · Research methods in applied linguistics · doi:10.1075/rmal.2 · Cited by 32
Schillinger, D., McNamara, D. S., Crossley, S. A., Lyles, C. R., Moffet, H. H., Sarkar, U., Duran, N. D., Allen, J. Y., Liu, J. Y., Oryn, D. E., et al. (2017) · Journal of Diabetes Research · doi:10.1155/2017/1348242 · Cited by 30
“Coh-Metrix indices related to text cohesion, lexical sophistication, and syntactic complexity have been used to develop readability measures [50, 61–63].”
BackgroundMethodInfluentialSelf-citationFlorit, E., Carli, P. D., Lavelli, M., & Masón, L. (2022) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12754 · Cited by 32
Learning linkages: Integrating data streams of multiple modalities and timescales
Liu, R., Stamper, J. C., Davenport, J. L., Crossley, S. A., McNamara, D. S., Nzinga, K. L. K., & Sherin, B. L. (2018) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12315 · Cited by 28
“NLP has the potential to provide information about language at multiple levels and dimensions (Graesser & McNamara, 2011).”
BackgroundSelf-citationGoldman, S. R., & Lee, C. D. (2014) · The Elementary School Journal · doi:10.1086/678298 · Cited by 36
“…in this special issue, every tool referenced in the articles takes a big-grain size view of genre, differentiating among them at what boils down to two categories: narrative versus informational, with some distinguishing science from other disciplinary texts (e.g., Graesser & McNamara, 2011).”
MethodResultEffects of disfluency in writing
Medimorec, S., & Risko, E. F. (2016) · British Journal of Psychology · doi:10.1111/bjop.12177 · Cited by 27
“Finally, cohesive devices indicate the degree to which concepts in a text are linked (Graesser & McNamara, 2011).”
BackgroundCoh-Metrix Model-Based Automatic Assessment of Interpreting Quality
Ouyang, L., Lv, Q., & Liang, J. (2021) · New frontiers in translation studies · doi:10.1007/978-981-15-8554-8_9 · Cited by 26
Modeling Learners’ Cognitive, Affective, and Social Processes through Language and Discourse
Dowell, N., & Graesser, A. C. (2014) · Journal of Learning Analytics · doi:10.18608/10.18608/jla.2014.13.18 · Cited by 26
“Psychological frameworks of comprehension and learning have identified the representations, structures, strategies, and processes at multiple levels of discourse (Graesser & McNamara, 2011; Kintsch, 1998; Snow, 2002).”
BackgroundSelf-citationThe Testing Effect: An Intervention on Behalf of Low-Skilled Comprehenders in General Chemistry
Pyburn, D. T., Pazicni, S., Benassi, V. A., & Tappin, E. E. (2014) · Journal of Chemical Education · doi:10.1021/ed4009045 · Cited by 26
The narrative waltz: The role of flexibility in writing proficiency.
Allen, L. K., Snow, E. L., & McNamara, D. S. (2016) · Journal of Educational Psychology · doi:10.1037/edu0000109 · Cited by 26
Self-citationMcCarthy, K. S. (2015) · Scientific Study of Literature · doi:10.1075/ssol.5.1.05mcc · Cited by 25
Booth, C. R., Brown, H. L., Eason, E. G., Wallot, S., & Kelty‐Stephen, D. G. (2016) · Discourse Processes · doi:10.1080/0163853x.2016.1197811 · Cited by 25
“…words not primed by previous long-range narrative structure and requiring greater processing effort resulting in increased word reading times (RTs), reflecting a reassessment of the text and of reader expectations (Blanchard & Iran-Nejad, 1987; Graesser & McNamara, 2011; Hoeken & van Vliet, 2000).”
BackgroundTamang, L. J., Alshaikh, Z., Khayi, N. A., Oli, P., & Rus, V. (2021) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/3408877.3432423 · Cited by 25
“This challenge is not surprising given that constructing mental representations is considered a higher-level skill of comprehension, typically engendering a high cognitive load[20, 25, 48, 55].”
BackgroundPatterns of Adults with Low Literacy Skills Interacting with an Intelligent Tutoring System
Fang, Y., Lippert, A. M., Cai, Z., Chen, S., Frijters, J. C., Greenberg, D., & Graesser, A. C. (2021) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-021-00266-y · Cited by 24
Self-citationWhat’s in the brain that ink may character ….
Jacobs, A. M., Schuster, S., Xue, S., & Lüdtke, J. (2017) · Scientific Study of Literature · doi:10.1075/ssol.7.1.02jac · Cited by 28
“However, text difficulty can be assessed at both a broader and deeper level than in equation (1) by using the eight text easability principal component z-scores of Coh-metrix (cf. Graesser & McNamara, 2010; Graesser et al., 2011; McNamara, Louwerse, McCarthy, & Graesser, 2010).”
BackgroundSchurer, T., Opitz, B., & Schubert, T. (2023) · Acta Psychologica · doi:10.1016/j.actpsy.2023.103836 · Cited by 28
Golke, S., Hagen, R., & Wittwer, J. (2018) · Learning and Instruction · doi:10.1016/j.learninstruc.2018.11.003 · Cited by 23
Chapter 4. Discourse performance in L2 task repetition
Wang, Z., & Chen, G.-W. (2018) · Task-Based Language Teaching · doi:10.1075/tblt.11.04wan · Cited by 5
A Lightweight Regression Method to Infer Psycholinguistic Properties for Brazilian Portuguese
dos Santos, L. B., Duran, M. S., Hartmann, N. S., Cândido, A., Paetzold, G. H., & Aluísio, S. M. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-64206-2_32 · Cited by 22
“Besides that, the logarithm of word frequency was used here because reading times are linearly related to the logarithm of word frequency, not to raw word frequencies [1].”
MethodYang, X. H., Zhang, X., Yang, Y., & Lin, N. (2018) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2018.01.010 · Cited by 19
Predicting Learning in a Multi-component Serious Game
Forsyth, C. M., Graesser, A. C., & Millis, K. (2019) · Technology Knowledge and Learning · doi:10.1007/s10758-019-09421-w · Cited by 22
Self-citationA New Measure of Text Formality: An Analysis of Discourse of Mao Zedong
Li, H., Graesser, A. C., Conley, M. W., Cai, Z., Pavlik, P. I., & Pennebaker, J. W. (2015) · Discourse Processes · doi:10.1080/0163853x.2015.1010191 · Cited by 22
“Recently, Graesser et al. (2014) proposed a measure of formality that considered the levels of the multilevel theoretical framework of discourse comprehension proposed by Graesser and McNamara (2011).”
BackgroundSelf-citationAutomated Disengagement Tracking Within an Intelligent Tutoring System
Su, C., Fang, Y., Shi, G., Sabatini, J., Greenberg, D., Frijters, J. C., & Graesser, A. C. (2021) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2020.595627 · Cited by 24
“The AutoTutor lessons were also scaled on four theoretical levels: Words (W), textbase (TB), Referential Situation Model (RSM), and Rhetorical Structure (RS), based on Graesser and McNamara ’ s (2011) multilevel theoretical framework.”
MethodSelf-citationDeveloping pedagogically-guided threshold algorithms for intelligent automated essay feedback
Roscoe, R. D., Kugler, D., Crossley, S. A., Weston, J. L., & McNamara, D. S. (2012) · The Florida AI Research Society · Cited by 21
Self-citationA corpus-based search for machine translationese in terms of discourse coherence
Jiang, Y., & Niu, J. (2022) · Across Languages and Cultures · doi:10.1556/084.2022.00182 · Cited by 20
The language of instruction: Compensating for challenge in lectures.
Medimorec, S., Pavlik, P. I., Olney, A. M., Graesser, A. C., & Risko, E. F. (2015) · Journal of Educational Psychology · doi:10.1037/edu0000024 · Cited by 19
Self-citationEducational Technologies that Support Reading Comprehension for Adults Who Have Low Literacy Skills
Graesser, A. C., Greenberg, D., Olney, A. M., & Lovett, M. W. (2019) · doi:10.1002/9781119261407.ch22 · Cited by 20
Self-citationYang, Y., Wei, X., Li, P., & Zhai, X. (2023) · ReCALL · doi:10.1017/s0958344023000022 · Cited by 25
“It is a robust tool, which can help researchers “ acquire a deeper understanding of language-discourse constraints ” (Graesser & McNamara, 2011: 372).”
BackgroundWhy does decomposed audit proposal readability differ by audit firm size? A Coh-Metrix approach
Chang, Y.-T., & Stone, D. N. (2019) · Managerial Auditing Journal · doi:10.1108/maj-02-2018-1789 · Cited by 19
Comprehension of Short Stories: Effects of Task Instructions on Literary Interpretation
McCarthy, K. S., & Goldman, S. R. (2014) · Discourse Processes · doi:10.1080/0163853x.2014.967610 · Cited by 23
Li, H., Xing, W., Li, C., Zhu, W., & Oh, H. (2024) · British Journal of Educational Technology · doi:10.1111/bjet.13554 · Cited by 23
Data Augmentation for Sparse Multidimensional Learning Performance Data Using Generative AI
Zhang, L., Lin, J., Sabatini, J., Borchers, C., Weitekamp, D., Cao, M., Hollander, J., Hu, X., & Graesser, A. C. (2025) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2025.3526582 · Cited by 23
“The number of features identified, averaging around six, closely aligns with Graesser–McNamara’s multilevel theoretical framework [85] Constructing a 3-”
Self-citationForsyth, C. M., Graesser, A. C., Pavlik, P. I., Cai, Z., Butler, H. A., Halpern, D. F., & Millis, K. (2013) · Journal of Educational Data Mining · doi:10.5281/zenodo.3554615 · Cited by 17
“Many more analyses could be conducted on the text content, such as the measures derived from CohMetrix [Graesser and McNamara 2011].”
MethodSelf-citation
Metadata
- Authors
- Graesser, A. C., & McNamara, D. S.
- Affiliations
- University of Memphis (US)
- Corresponding author
- Graesser, Arthur C.
- Venue
- Topics in Cognitive Science
- Publisher
- Wiley
- Published
- 2011-04
- Volume
- 3
- Issue
- 2
- Pages
- 371-398
- ISSN
- 1756-8757 (ISSN-L)
- 1756-8765
- OpenAlex
- W2087016971
- Cited by
- 325
- References
- 115
- Citations by year
- 2026: 8 · 2025: 13 · 2024: 13 · 2023: 24 · 2022: 46 · 2021: 27 · 2020: 22 · 2019: 22 · 2018: 17 · 2017: 13 · 2016: 23 · 2015: 28 · 2014: 27 · 2013: 18 · 2012: 15
- FWCI
- 3.02
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Topics in Cognitive Science (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Science Foundation (0904909, DRK12, 0918409, 0834847, 0325428)
- U.S. Department of Defense (H9C104-07-0014)
- Institute of Education Sciences (R305G020018, R305A080594, R305B070349, R305A080589, R305H050169)
- Topics
- Social Sciences › Linguistics and Language › Text Readability and Simplification
- Social Sciences › Psychology › Developmental and Educational Psychology › Reading and Literacy Development
- Social Sciences › Linguistics and Language › Linguistics and Discourse Analysis
- Keywords
- Coh-Metrix, textual cohesion, situation model, textbase, pragmatic communication, narrative texts, scientific texts
- MeSH terms
- Artificial Intelligence, Cognition, Humans, Linguistics, Reading, Semantics, User-Computer Interface, Comprehension
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
