Null Effects of Perceptual Disfluency on Learning Outcomes in a Text-Based Educational Context: a Meta-analysis
The DOI matches its Crossref record; 83 of the 83 references it cites were found in a registry or an index.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Although several results in the field of disfluency research have been put into question (e.g., Xie et al., 2018), there are a number of results using more subtle disfluent cues that appear to more consistently lead to an increased learning performance (or, at least, not to significant disadvantages).”
From Understanding Cognitive Load in Digital and Online Learning: a New Perspective on Extraneous Cognitive Load (2022) · cites this paper as Xie et al., 2018
Citations
Cites
Statistical Power Analysis for the Behavioral Sciences
Muller, K. (1989) · Technometrics · doi:10.1080/00401706.1989.10488618 · Cited by 99,167
Statistical power analysis for the behavioral sciences
(1990) · Computers, Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,269
Measuring inconsistency in meta-analyses
Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003) · BMJ · doi:10.1136/bmj.327.7414.557 · Cited by 63,139
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722
Statistical Power Analysis for the Behavioral Sciences (2nd ed.).
Lachenbruch, P. A., & Cohen, J. (1989) · Journal of the American Statistical Association · doi:10.2307/2290095 · Cited by 37,395
James, W. (1913) · Henry Holt and Company eBooks · doi:10.1037/11059-000 · Cited by 25,226
Operating characteristics of a rank correlation test for publication bias
Begg, C. B., & Mazumdar, M. (1994) · Biometrics · doi:10.2307/2533446 · Cited by 16,955
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009) · doi:10.1002/9780470743386 · Cited by 14,225
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2021) · doi:10.1002/9781119558378 · Cited by 9,799
(2000) · Cited by 9,198
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
Cohen, J. (1992) · Psychological Bulletin · doi:10.1037//0033-2909.112.1.155 · Cited by 8,771
Mayer, R. E. (2009) · doi:10.1017/cbo9780511811678 · Cited by 7,800
Mayer, R. (2020) · doi:10.1017/9781316941355 · Cited by 7,777
Cognitive Architecture and Instructional Design
Sweller, J., van Merrienboer, J. J. G., & Paas, F. G. W. C. (1998) · Educational Psychology Review · doi:10.1023/a:1022193728205 · Cited by 6,023
The Psychology of Learning and Motivation: Advances in Research and Theory
(2010) · Psychology of Learning and Motivation · doi:10.1016/c2009-0-62209-1 · Cited by 5,994
Sweller, J., Ayres, P., & Kalyuga, S. (2011) · doi:10.1007/978-1-4419-8126-4 · Cited by 5,531
Baddeley, A. (1992) · Science · doi:10.1126/science.1736359 · Cited by 5,258
Cognitive Reflection and Decision Making
Frederick, S. (2005) · Journal of Economic Perspectives · doi:10.1257/089533005775196732 · Cited by 5,256
Sweller, J. (2011) · Psychology of Learning and Motivation · doi:10.1016/b978-0-12-387691-1.00002-8 · Cited by 4,856
Sweller, J. (2012) · Encyclopedia of the Sciences of Learning · doi:10.1007/978-1-4419-1428-6_446 · Cited by 4,405
Uniting the Tribes of Fluency to Form a Metacognitive Nation
Alter, A. L., & Oppenheimer, D. M. (2009) · Personality and Social Psychology Review · doi:10.1177/1088868309341564 · Cited by 1,995
Kalyuga, S., Ayres, P., Chandler, P., & Sweller, J. (2003) · Educational Psychologist · doi:10.1207/s15326985ep3801_4 · Cited by 1,927
Metacognition: Knowing about Knowing
(1994) · The MIT Press eBooks · doi:10.7551/mitpress/4561.001.0001 · Cited by 1,720
TVERSKY, B., MORRISON, J. B., & BETRANCOURT, M. (2002) · International Journal of Human-Computer Studies · doi:10.1006/ijhc.2002.1017 · Cited by 1,671
Publication bias in the social sciences: Unlocking the file drawer
Franco, A., Malhotra, N., & Simonovits, G. (2014) · Science · doi:10.1126/science.1255484 · Cited by 1,542 · cited 1×
“Studies have empirically confirmed the existence of publication bias (Franco et al. 2014; Rosenthal 1979).”
Monitoring one’s own knowledge during study: A cue-utilization approach to judgments of learning
Koriat, A. (1997) · Journal of Experimental Psychology: General · doi:10.1037/0096-3445.126.4.349 · Cited by 1,363
McNamara, D. S., Kintsch, E., Songer, N. B., & Kintsch, W. (1996) · Cognition and Instruction · doi:10.1207/s1532690xci1401_1 · Cited by 1,341 · cited 1×
“…we included studies where disfluency was manipulated on a perceptual level but not on a conceptual level (e.g., by altering text coherence; McNamara et al. 1996), and another interesting direction for future research would be to investigate the different effects of perceptual disfluency…”
BackgroundMultimedia Learning: SUBJECT INDEX
Mayer, R. E. (2001) · Cited by 1,175
Memory and Metamemory Considerations in the Training of Human Beings
Bjork, R. A. (1994) · Metacognition · doi:10.7551/mitpress/4561.003.0011 · Cited by 1,084
Expertise Reversal Effect and Its Implications for Learner-Tailored Instruction
Kalyuga, S. (2007) · Educational Psychology Review · doi:10.1007/s10648-007-9054-3 · Cited by 1,056
Overcoming intuition: Metacognitive difficulty activates analytic reasoning.
Alter, A. L., Oppenheimer, D. M., Epley, N., & Eyre, R. N. (2007) · Journal of Experimental Psychology: General · doi:10.1037/0096-3445.136.4.569 · Cited by 844 · cited 7×
“The activation of systems 1 and 2 depends on the subjective sense of ease or difficulty of a cognitive task (Alter et al. 2007).”
BackgroundMethodInfluentialAtkinson, R. K., Renkl, A., & Merrill, M. M. (2003) · Journal of Educational Psychology · doi:10.1037/0022-0663.95.4.774 · Cited by 491
Metacognitive and control strategies in study-time allocation.
Son, L. K., & Metcalfe, J. (2000) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037//0278-7393.26.1.204 · Cited by 466
Koriat, A., Ma'ayan, H., & Nussinson, R. (2006) · Journal of Experimental Psychology: General · doi:10.1037/0096-3445.135.1.36 · Cited by 422
Evidence that judgments of learning are causally related to study choice
METCALFE, J., & FINN, B. (2008) · Psychonomic Bulletin & Review · doi:10.3758/pbr.15.1.174 · Cited by 409
Fortune favors the ( ): Effects of disfluency on educational outcomes
Diemand-Yauman, C., Oppenheimer, D. M., & Vaughan, E. B. (2011) · Cognition · doi:10.1016/j.cognition.2010.09.012 · Cited by 321
Mayer, R. E. (1984) · Educational Psychologist · doi:10.1080/00461528409529279 · Cited by 312
Optimising worked example instruction: Different ways to increase germane cognitive load
Paas, F., & van Gog, T. (2006) · Learning and Instruction · doi:10.1016/j.learninstruc.2006.02.004 · Cited by 252
The generation effect: Support for a two-factor theory.
Hirshman, E., & Bjork, R. A. (1988) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037//0278-7393.14.3.484 · Cited by 251
Reber, R., Wurtz, P., & Zimmermann, T. D. (2004) · Consciousness and Cognition · doi:10.1016/s1053-8100(03)00049-7 · Cited by 247
Thompson, V. A., Turner, J. A. P., Pennycook, G., Ball, L. J., Brack, H., Ophir, Y., & Ackerman, R. (2013) · Cognition · doi:10.1016/j.cognition.2012.09.012 · Cited by 215
Yue, C. L., Castel, A. D., & Bjork, R. A. (2013) · Memory & Cognition · doi:10.3758/s13421-012-0255-8 · Cited by 204
Mueller, M. L., Dunlosky, J., Tauber, S. K., & Rhodes, M. G. (2014) · Journal of Memory and Language · doi:10.1016/j.jml.2013.09.007 · Cited by 174
Improving self-monitoring and self-regulation: From cognitive psychology to the classroom
de Bruin, A. B. H., & van Gog, T. (2012) · Learning and Instruction · doi:10.1016/j.learninstruc.2012.01.003 · Cited by 173
Self-Regulated Learning and the Allocation of Study Time
Dunlosky, J. T., & Ariel, R. (2011) · The Psychology of learning and motivation/The psychology of learning and motivation · doi:10.1016/b978-0-12-385527-5.00004-8 · Cited by 171
Carpenter, S. K., Wilford, M. M., Kornell, N., & Mullaney, K. M. (2013) · Psychonomic Bulletin & Review · doi:10.3758/s13423-013-0442-z · Cited by 165
Naive theories of intelligence and the role of processing fluency in perceived comprehension.
Miele, D. B., & Molden, D. C. (2010) · Journal of Experimental Psychology: General · doi:10.1037/a0019745 · Cited by 127
The relationships between monitoring, regulation and performance
Koriat, A. (2012) · Learning and Instruction · doi:10.1016/j.learninstruc.2012.01.002 · Cited by 102
Besken, M., & Mulligan, N. W. (2013) · Memory & Cognition · doi:10.3758/s13421-013-0307-8 · Cited by 95
Instructional benefits of spoken words: A review of cognitive load factors
Kalyuga, S. (2012) · Educational Research Review · doi:10.1016/j.edurev.2011.12.002 · Cited by 93
Katzir, T., Hershko, S., & Halamish, V. (2013) · PLoS ONE · doi:10.1371/journal.pone.0074061 · Cited by 89
Reber, R., & Greifeneder, R. (2017) · Educational Psychologist · doi:10.1080/00461520.2016.1258173 · Cited by 87
Eitel, A., Kühl, T., Scheiter, K., & Gerjets, P. (2014) · Applied Cognitive Psychology · doi:10.1002/acp.3004 · Cited by 85
Processing strategies and the generation effect: Implications for making a better reader
DeWinstanley, P. A., & Bjork, E. L. (2004) · Memory & Cognition · doi:10.3758/bf03196872 · Cited by 83
Disfluent fonts don’t help people solve math problems.
Meyer, A., Frederick, S., Burnham, T. C., Guevara Pinto, J. D., Boyer, T. W., Ball, L. J., Pennycook, G., Ackerman, R., Thompson, V. A., & Schuldt, J. P. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000049 · Cited by 79
Does the Modality Effect Exist? and if So, Which Modality Effect?
Reinwein, J. (2011) · Journal of Psycholinguistic Research · doi:10.1007/s10936-011-9180-4 · Cited by 77
Effects of disfluency on cognitive and metacognitive processes and outcomes
Kühl, T., & Eitel, A. (2016) · Metacognition and Learning · doi:10.1007/s11409-016-9154-x · Cited by 74
Pieger, E., Mengelkamp, C., & Bannert, M. (2016) · Learning and Instruction · doi:10.1016/j.learninstruc.2016.01.012 · Cited by 74
Is disfluency desirable for learning?
Weißgerber, S. C., & Reinhard, M. (2017) · Learning and Instruction · doi:10.1016/j.learninstruc.2017.02.004 · Cited by 64
The impact of disfluency, pacing, and students’ need for cognition on learning with multimedia
Kühl, T., Eitel, A., Damnik, G., & Körndle, H. (2014) · Computers in Human Behavior · doi:10.1016/j.chb.2014.03.004 · Cited by 62
Working memory capacity and disfluency effect: an aptitude-treatment-interaction study
Lehmann, J., Goussios, C., & Seufert, T. (2016) · Metacognition and Learning · doi:10.1007/s11409-015-9149-z · Cited by 61
The effects of different levels of disfluency on learning outcomes and cognitive load
Seufert, T., Wagner, F., & Westphal, J. (2017) · Instructional Science · doi:10.1007/s11251-016-9387-8 · Cited by 56
Changing Fonts in Education: How the Benefits Vary with Ability and Dyslexia
French, M. M. J., Blood, A., Bright, N. D., Futak, D., Grohmann, M. J., Hasthorpe, A., Heritage, J., Poland, R. L., Reece, S., & Tabor, J. (2013) · The Journal of Educational Research · doi:10.1080/00220671.2012.736430 · Cited by 52
Fortune is fickle: null-effects of disfluency on learning outcomes
Rummer, R., Schweppe, J., & Schwede, A. (2016) · Metacognition and Learning · doi:10.1007/s11409-015-9151-5 · Cited by 51
Sidi, Y., Ophir, Y., & Ackerman, R. (2016) · Metacognition and Learning · doi:10.1007/s11409-015-9150-6 · Cited by 46
Effects of disfluency and test expectancy on learning with text
Eitel, A., & Kühl, T. (2016) · Metacognition and Learning · doi:10.1007/s11409-015-9145-3 · Cited by 42
Further boundary conditions for the effects of perceptual disfluency on judgments of learning
Magreehan, D. A., Serra, M. J., Schwartz, N. H., & Narciss, S. (2016) · Metacognition and Learning · doi:10.1007/s11409-015-9147-1 · Cited by 40
Carpenter, S. K., Mickes, L., Rahman, S., & Fernandez, C. (2016) · Journal of Experimental Psychology Applied · doi:10.1037/xap0000077 · Cited by 40
The Search for Moderators in Disfluency Research
Oppenheimer, D. M., & Alter, A. L. (2014) · Applied Cognitive Psychology · doi:10.1002/acp.3023 · Cited by 36
Instructor accents in online education and their effect on learning and attitudes
Sanchez, C. A., & Khan, S. H. (2016) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12149 · Cited by 34
The effect of disfluency on mind wandering during text comprehension
Faber, M., Mills, C., Kopp, K., & D’Mello, S. (2017) · Psychonomic Bulletin & Review · doi:10.3758/s13423-016-1153-z · Cited by 31
Desirable Difficulties Perspective on Learning
(2013) · Encyclopedia of the Mind · doi:10.4135/9781452257044.n88 · Cited by 30
Dunlosky, J. T., & Mueller, M. L. (2016) · Metacognition and Learning · doi:10.1007/s11409-016-9155-9 · Cited by 29
Kühl, T., Eitel, A., Scheiter, K., & Gerjets, P. H. (2014) · Applied Cognitive Psychology · doi:10.1002/acp.3030 · Cited by 27
Exploring the lack of a disfluency effect: evidence from eye movements
Strukelj, A., Scheiter, K., Nyström, M., & Holmqvist, K. (2016) · Metacognition and Learning · doi:10.1007/s11409-015-9146-2 · Cited by 26
Commentary: Is disfluency desirable?
Bjork, R. A., & Yue, C. L. (2016) · Metacognition and Learning · doi:10.1007/s11409-016-9156-8 · Cited by 25
Pieger, E., Mengelkamp, C., & Bannert, M. (2017) · Applied Cognitive Psychology · doi:10.1002/acp.3326 · Cited by 23
Ball, B. H., Klein, K. N., & Brewer, G. A. (2014) · Journal of Experimental Psychology: Applied · doi:10.1037/xap0000023 · Cited by 18
Does the Processing Fluency of a Syllabus Affect the Forecasted Grade and Course Difficulty?
Guenther, R. K. (2012) · Psychological Reports · doi:10.2466/01.11.28.pr0.110.3.946-954 · Cited by 17
Weltman, D., & Eakin, M. (2014) · INFORMS Transactions on Education · doi:10.1287/ited.2014.0130 · Cited by 14
Effects of Disfluent Kanji Fonts on Reading Retention with E-Book
Lee, M.-H. (2013) · IEEE International Conference on Advanced Learning Technologies (ICALT) · doi:10.1109/icalt.2013.152 · Cited by 10
Does harder-to-read mean better-to-learn? Effects of disfluency on learning outcomes
Xie, H., Fuxing, W. A. N. G., Yuxin, W. A. N. G., & An, J. (2016) · Advances in Psychological Science · doi:10.3724/sp.j.1042.2016.01077 · Cited by 6
Self-citation
Research works printed but not linked: 3
- Dunlosky, J., & Metcalfe, J. (2009). Metacognition. Los Angeles: SAGE.
- Haysom, H. J. (2012). Could Comic Sans make you smarter? An exploration of the effects of disfluency on learning outcomes (Honours thesis). Brisbane: University of Queensland This study is included in the meta-analysis.
- Whitehouse, E. (2011). An investigation into disfluency effects: depth of learning and affective outcomes (Honours thesis). Brisbane: University of Queensland. This study is included in the meta-analysis.
Cited by
Skulmowski, A., & Xu, K. M. (2022) · Educational Psychology Review · doi:10.1007/s10648-021-09624-7 · Cited by 678
Xie, H., Peng, J., Qin, M., Huang, X., Tian, F., & Zhou, Z. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.02580 · Cited by 108
“These sets of characteristics are usually regarded as important potential moderators by researchers in the field of learning and instruction (e.g., Berney and Bétrancourt, 2016; Fiorella and Zhang, 2018; Xie et al., 2018).”
BackgroundSelf-citationRenkewitz, F., & Keiner, M. (2019) · Zeitschrift für Psychologie · doi:10.1027/2151-2604/a000386 · Cited by 55
Skulmowski, A., & Rey, G. D. (2020) · Human Behavior and Emerging Technologies · doi:10.1002/hbe2.190 · Cited by 50
Clinton‐Lisell, V. (2021) · Journal of Research in Reading · doi:10.1111/1467-9817.12372 · Cited by 55
Metadata
- Authors
- Xie, H., Zhou, Z., & Liu, Q.
- Affiliations
- Central China Normal University (CN)
- Corresponding author
- Zhou, Zongkui
- Venue
- Educational Psychology Review
- Published
- 2018-06-20
- Volume
- 30
- Issue
- 3
- Pages
- 745-771
- ISSN
- 1040-726X (ISSN-L)
- 1573-336X
- OpenAlex
- W2809417050
- Cited by
- 46
- References
- 81
- Citations by year
- 2026: 4 · 2025: 4 · 2024: 4 · 2023: 6 · 2022: 7 · 2021: 9 · 2020: 5 · 2019: 2 · 2018: 4
- FWCI
- 0.80
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Educational Psychology Review (publishedVersion)
- Funders
- Major Program of National Social Science Foundation of China (11&ZD151)
- Fundamental Research Funds of Central China Normal University (CCNU14Z02004)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Visual and Cognitive Learning Processes
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory Processes and Influences
- Social Sciences › Psychology › Developmental and Educational Psychology › Reading and Literacy Development
- Keywords
- text-based learning, memory retrieval, transfer, judgments of learning, learning time, metacognition, cognitive load, analytical processing, reading difficulties, learning outcomes
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.