Assessing Efficiency of Prompts Based on Learner Characteristics
Open accessThe DOI matches its Crossref record; 49 of the 49 references it cites were found in a registry or an index.
Abstract
Personalized prompting research has shown the significant learning benefit of prompting. The current paper outlines and examines a personalized prompting approach aimed at eliminating performance differences on the basis of a number of learner characteristics (capturing learning strategies and traits). The learner characteristics of interest were the need for cognition, work effort, computer self-efficacy, the use of surface learning, and the learner’s confidence in their learning. The approach was tested in two e-modules, using similar assessment forms (experimental n = 413; control group n = 243). Several prompts which corresponded to the learner characteristics were implemented, including an explanation prompt, a motivation prompt, a strategy prompt, and an assessment prompt. All learning characteristics were significant correlates of at least one of the outcome measures (test performance, errors, and omissions). However, only the assessment prompt increased test performance. On this basis, and drawing upon the testing effect, this prompt may be a particularly promising option to increase performance in e-learning and similar personalized systems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, delivering personalised scaffolds automatically during the student task requires the formulation of rules that map insights from LA to actionable suggestions for specific learning tasks (Backhaus et al., 2017; Schumacher and Ifenthaler, 2021; Li et al., 2023; Liu et al., 2024b; Gunawardena et al., 2024; Li et al., 2025).”
From Towards reliable generative AI-driven scaffolding: Reducing hallucinations and enhancing quality in self-regulated learning support (2026) · cites this paper as Backhaus et al., 2017
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Computer Self-Efficacy: Development of a Measure and Initial Test
Compeau, D. R., & Higgins, C. A. (1995) · MIS Quarterly · doi:10.2307/249688 · Cited by 6,553
Flesch, R. (1948) · Journal of Applied Psychology · doi:10.1037/h0057532 · Cited by 5,391 · cited 5×
“The ARI and the FRERF are formulae to measure the understandability of texts.”
InfluentialThe Role of Goal Orientation in Self-Regulated Learning
Pintrich, P. R. (2000) · Handbook of Self-Regulation · doi:10.1016/b978-012109890-2/50043-3 · Cited by 5,173 · cited 1×
“Prompts can serve multiple functions and can focus on cognitive learning processes, metacognitive self-regulation, and resource management [17].”
BackgroundThe Efficient Assessment of Need for Cognition
Cacioppo, J. T., Petty, R. E., & Feng Kao, C. (1984) · Journal of Personality Assessment · doi:10.1207/s15327752jpa4803_13 · Cited by 2,913
Test-Enhanced Learning: Taking Memory Tests Improves Long-Term Retention
Roediger, H. L., & Karpicke, J. D. (2006) · Psychological Science · doi:10.1111/j.1467-9280.2006.01693.x · Cited by 2,867
Cacioppo, J. T., Petty, R. E., Feinstein, J. A., & Jarvis, W. B. G. (1996) · Psychological Bulletin · doi:10.1037/0033-2909.119.2.197 · Cited by 2,517 · cited 1×
“Greater need for cognition means the learner is motivated to try to understand new information [8], while those with lower scores are not driven to learn for the sake of gaining new insight—in which case a prompt may be helpful.”
The revised two‐factor Study Process Questionnaire: R‐SPQ‐2F
Biggs, J., Kember, D., & Leung, D. Y. P. (2001) · British Journal of Educational Psychology · doi:10.1348/000709901158433 · Cited by 2,122 · cited 1×
“Surface strategy was measured using three items of the two-factor Study Process Questionnaire [9].”
Cohen, J. (1983) · Applied Psychological Measurement · doi:10.1177/014662168300700301 · Cited by 1,564 · cited 1×
“learning characteristic reduces variance [43].”
Self-regulated learning : from teaching to self-reflective practice
Schunk, D. H., & Zimmerman, B. J. (1998) · Cited by 1,426
Moreno, R., Mayer, R. E., Spires, H. A., & Lester, J. C. (2001) · Cognition and Instruction · doi:10.1207/s1532690xci1902_02 · Cited by 912 · cited 1×
“Moreno, Mayer, Spires, and Lester employed a prompt that required students to feedback how much the learning material presented to them also helped them understand the link between plant design and the environment [19].”
Sitzmann, T., & Ely, K. (2011) · Psychological Bulletin · doi:10.1037/a0022777 · Cited by 897 · cited 2×
“This may also improve self-regulation and, thus, improve performance [20].”
BackgroundDunning, D., Meyerowitz, J. A., & Holzberg, A. D. (1989) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.57.6.1082 · Cited by 758 · cited 1×
“In addition, confidence assessments during the learning process (in form of a confidence prompt ) may also reduce overly optimistic learner expectations regarding their learning progress [28].”
Eliciting Self‐Explanations Improves Understanding
Chi, M. T. H., De Leeuw, N., Chiu, M., & Lavancher, C. (1994) · Cognitive Science · doi:10.1207/s15516709cog1803_3 · Cited by 757 · cited 1×
“In addition, they do not accurately estimate changes in mean performance related to task difficulty [13]; Third, the presentation of a metacognitive prompt may reduce the illusion of understanding that may arise in learners [14].”
Dunning, D., Meyerowitz, J. A., & Holzberg, A. D. (2002) · Heuristics and Biases · doi:10.1017/cbo9780511808098.020 · Cited by 678
A Goodness of Fit Test for the Rasch Model
Andersen, E. B. (1973) · Psychometrika · doi:10.1007/bf02291180 · Cited by 664
Smith, E. A., & Senter, R. J. (1967) · PubMed · Cited by 587
Test prediction and performance in a classroom context.
Hacker, D. J., Bol, L., Horgan, D. D., & Rakow, E. A. (2000) · Journal of Educational Psychology · doi:10.1037/0022-0663.92.1.160 · Cited by 464 · cited 2×
“A verification prompt can be useful here as it may reduce over-confidence or faking (reflected in extremely high scores on certain learning characteristics, such as work effort and need for cognition), as well as assist self-regulation in online settings [26,27].”
BackgroundKlare, G. R. (1974) · Reading Research Quarterly · doi:10.2307/747086 · Cited by 398
Devolder, A., van Braak, J., & Tondeur, J. (2012) · Journal of Computer Assisted Learning · doi:10.1111/j.1365-2729.2011.00476.x · Cited by 258 · cited 1×
“In the domain of science education, a number of reviews exist that provide further insight (e.g., [18]).”
Barbeite, F. G., & Weiss, E. M. (2004) · Computers in Human Behavior · doi:10.1016/s0747-5632(03)00049-9 · Cited by 255 · cited 2×
“Computer self-efficacy was measured using three items [35].”
CULTURAL SPECIFICITY OF APPROACHES TO STUDY
KEMBER, D., & GOW, L. (1990) · British Journal of Educational Psychology · doi:10.1111/j.2044-8279.1990.tb00952.x · Cited by 239 · cited 1×
“To avoid socially desirable responses all items were reverse-coded [37].”
BackgroundBerthold, K., Eysink, T. H. S., & Renkl, A. (2009) · Instructional Science · doi:10.1007/s11251-008-9051-z · Cited by 238 · cited 1×
“In order to facilitate learning, several different and very specific prompts may be employed that focus on problem-solving [15] general instruction [5], self-explanation [16], or encouraging recall, reasoning, or observation [5].”
BackgroundSupporting self-regulated hypermedia learning through prompts
Bannert, M., & Reimann, P. (2012) · Instructional Science · doi:10.1007/s11251-011-9167-4 · Cited by 213 · cited 2×
“Such guidance may come in numerous forms, including prompts that enable the learner to regulate all of those (cognitive, motivational, or behavioral) processes that are particular to the individual learner’s approach [6].”
Individual Differences in Absolute and Relative Metacomprehension Accuracy.
Maki, R. H., Shields, M., Wheeler, A. E., & Zacchilli, T. L. (2005) · Journal of Educational Psychology · doi:10.1037/0022-0663.97.4.723 · Cited by 207 · cited 2×
“Overconfidence is particularly an issue for over-confident individuals who actually have lower ability and performance [13,26], as such overconfidence may also affect progression and effort.”
BackgroundEffects of high level prompts and peer assessment on online learners’ reflection levels
Chen, N.-S., Wei, C.-W., Wu, K.-T., & Uden, L. (2009) · Computers & Education · doi:10.1016/j.compedu.2008.08.007 · Cited by 199 · cited 2×
“Prompts can help learners to engage more with the material but also reflect on their learning progress [4].”
Automatic detection of learning styles for an e-learning system
Özpolat, E., & Akar, G. B. (2009) · Computers & Education · doi:10.1016/j.compedu.2009.02.018 · Cited by 188 · cited 1×
“To date, many user and learner models personalize the learning experience by implementing tutoring support based on perceived or self-reported learner characteristics [2,3].”
Computer-supported example-based learning: When instructional explanations reduce self-explanations
Schworm, S., & Renkl, A. (2006) · Computers & Education · doi:10.1016/j.compedu.2004.08.011 · Cited by 175 · cited 2×
“Self-evaluations have been shown to increase performance in computer-supported learning [25].”
BackgroundChen, C.-M. (2009) · Expert Systems with Applications · doi:10.1016/j.eswa.2008.11.026 · Cited by 166 · cited 1×
“One of the challenges in personalized e-learning initiatives pertains to the selection and consideration of tools based on learner characteristics that may increase performance for different types of learners [1].”
BackgroundDemetriadis, S., Papadopoulos, P. M., Stamelos, I. G., & Fischer, F. (2008) · Computers & Education · doi:10.1016/j.compedu.2007.09.012 · Cited by 162 · cited 2×
“In order to facilitate learning, several different and very specific prompts may be employed that focus on problem-solving [15] general instruction [5], self-explanation [16], or encouraging recall, reasoning, or observation [5].”
Testing versus review: Effects on retention.
Nungester, R. J., & Duchastel, P. C. (1982) · Journal of Educational Psychology · doi:10.1037//0022-0663.74.1.18 · Cited by 144 · cited 2×
“This finding builds on the ideas around the testing effect [21], that by having learners recall what they have learned, they will be able to recall information better [22].”
Sitzmann, T., & Ely, K. (2010) · Journal of Applied Psychology · doi:10.1037/a0018080 · Cited by 141
Overcoming screen inferiority in learning and calibration
Lauterman, T., & Ackerman, R. (2014) · Computers in Human Behavior · doi:10.1016/j.chb.2014.02.046 · Cited by 138 · cited 1×
“A verification prompt can be useful here as it may reduce over-confidence or faking (reflected in extremely high scores on certain learning characteristics, such as work effort and need for cognition), as well as assist self-regulation in online settings [26,27].”
BackgroundKauffman, D. F., Ge, X., Xie, K., & Chen, C.-H. (2008) · Journal of Educational Computing Research · doi:10.2190/ec.38.2.a · Cited by 120 · cited 2×
“The latter was outside the remit of the current study but Kauffman and colleagues [15] may be a good starting point; And fifth, relying on null hypothesis testing [46] and adherence to p-values as the determinants of what may be considered meaningful has not only been criticized by leading statisticians [47] but may a…”
Understanding the Delayed-Keyword Effect on Metacomprehension Accuracy.
Thiede, K. W., Dunlosky, J., Griffin, T. D., & Wiley, J. (2005) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037/0278-7393.31.6.1267 · Cited by 116
Activity sequence modelling and dynamic clustering for personalized e-learning
Köck, M., & Paramythis, A. (2011) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-010-9087-z · Cited by 104 · cited 1×
“To date, many user and learner models personalize the learning experience by implementing tutoring support based on perceived or self-reported learner characteristics [2,3].”
BackgroundLearning Styles and Overall Academic Achievement in a Specific Educational System
Jafre, M., Abidin, Z., Rezaee, A. A., Abdullah, H. N., Kaur, K., & Singh, B. (2011) · Cited by 102
Development and Validation of the Work Effort Scale
De Cooman, R., De Gieter, S., Pepermans, R., Jegers, M., & Van Acker, F. (2009) · European Journal of Psychological Assessment · doi:10.1027/1015-5759.25.4.266 · Cited by 100 · cited 1×
“Work effort was measured using five items [34], the first three measuring persistence, the last two measuring work effort intensity.”
On the Past and Future of Null Hypothesis Significance Testing
Robinson, D. H., & Wainer, H. (2002) · Journal of Wildlife Management · doi:10.2307/3803158 · Cited by 99 · cited 1×
“The latter was outside the remit of the current study but Kauffman and colleagues [15] may be a good starting point; And fifth, relying on null hypothesis testing [46] and adherence to p-values as the determinants of what may be considered meaningful has not only been criticized by leading statisticians [47] but may a…”
Dimensions of the Need for Cognition: Subscales and Gender Differences
Tanaka, J. S., Panter, A. T., & Winborne, W. C. (1988) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr2301_2 · Cited by 92 · cited 1×
“Gender differences in self-rating have also been reported for need for cognition [41].”
BackgroundMartin, M. A. (2007) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2007.01.020 · Cited by 70 · cited 1×
“Group comparisons between the experimental and control group were conducted using an ANOVA bootstrap approach [42].”
MethodInfluences of metamemory on performance predictions for text
Rawson, K. A., Dunlosky, J., & McDonald, S. L. (2002) · The Quarterly Journal of Experimental Psychology Section A · doi:10.1080/02724980143000352 · Cited by 48
Self‐regulation during e‐learning: using behavioural evidence from navigation log files
Jeske, D., Backhaus, J., & Roßnagel, C. S. (2013) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12045 · Cited by 47
Self-citationKrause, U.-M., & Stark, R. (2010) · Evaluation & Research in Education · doi:10.1080/09500790.2010.519024 · Cited by 42 · cited 2×
“Krause and Stark [49] also observed no significant prompt difference in performance when they asked students to engage in active problem-solving with or without reflection prompts.”
Sanabria, F., & Killeen, P. R. (2007) · Psychology in the Schools · doi:10.1002/pits.20239 · Cited by 26 · cited 1×
“Sanabria and Killeen [52] suggested that replication statistics may be particularly helpful, and potentially more so than null hypothesis testing, in order to test replicated effects.”
Reisslein, J., Atkinson, R. K., Seeling, P., & Martin Reisslein, B. (2005) · IEEE Transactions on Education · doi:10.1109/te.2005.852602 · Cited by 21 · cited 1×
“For example, Reisslein [48] explored different prompt formats and presentation effects.”
BackgroundJeske, D., Roßnagel, C. S., & Backhaus, J. (2014) · The Journal of Interactive Learning Research · Cited by 15
Self-citationAppropriate Learning Resource Recommendation in Intelligent Web-Based Educational System
Daomin, X., & Ming-chui, D. (2013) · International Conference on Intelligent System Design and Engineering Application (ISDEA) · doi:10.1109/isdea.2013.443 · Cited by 12
Assessing Students’ Problem Solving Ability and Cognitive Regulation with Learning Trajectories
Stevens, R., Beal, C. R., & Sprang, M. (2013) · Springer international handbooks of education · doi:10.1007/978-1-4419-5546-3_27 · Cited by 11 · cited 1×
“An alternative approach could be to simply use percentile, or even decentile, rankings and create corresponding prompts or use an item response theoretical approach (for successful applications in online learning settings see work by [35,44,45]); Third, our information about participants was collected using self-repor…”
Jeske, D., Backhaus, J., & Roßnagel, C. S. (2013) · Learning and Individual Differences · doi:10.1016/j.lindif.2013.11.006 · Cited by 8 · cited 1×
“Thresholds were derived from data obtained in previous studies by the authors [38–40].”
Self-citation
Research works printed but not linked: 3
- Dunnette, M.D., and Hough, L.M. (1991). Handbook of Industrial and Organizational Psychology, Consulting Psychologists Press.
- Stevens, R., Beal, C.R., and Sprang, M. (2013). International Handbook of Metacognition and Learning Technologies, Springer.
- Gigerenzer, G., Krauss, S., and Vitouch, O. (2004). The Sage Handbook of Quantitative Methodology for the Social Sciences, Sage Publications.
Cited by
Guo, L. (2022) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12650 · Cited by 141
Schumacher, C., & Ifenthaler, D. (2021) · The Internet and Higher Education · doi:10.1016/j.iheduc.2020.100791 · Cited by 109
The Relation Between Need for Cognition and Academic Achievement: A Meta-Analysis
Liu, Q., & Nesbit, J. C. (2023) · Review of Educational Research · doi:10.3102/00346543231160474 · Cited by 88
Li, T., Nath, D., Cheng, Y., Fan, Y., Li, X., Raković, M., Khosravi, H., Swiecki, Z., Tsai, Y., & Gašević, D. (2025) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/3706468.3706559 · Cited by 50
Backhaus, J., Huth, K., Entwistle, A., Homayounfar, K., & Koenig, S. (2019) · Journal of surgical education · doi:10.1016/j.jsurg.2018.12.001 · Cited by 43
Self-citation
Metadata
- Authors
- Backhaus, J., Jeske, D., Poinstingl, H., & Koenig, S.
- Affiliations
- Universitätsklinikum Würzburg (DE)
- University College Cork (IE)
- University of Vienna (AT)
- Corresponding author
- Backhaus, Joy
- Venue
- Computers
- Publisher
- MDPI AG
- Published
- 2017-02-10
- Volume
- 6
- Issue
- 1
- Pages
- 7
- ISSN
- 2073-431X (ISSN-L)
- OpenAlex
- W2587652368
- Cited by
- 18
- References
- 49
- Citations by year
- 2026: 1 · 2025: 4 · 2023: 1 · 2022: 1 · 2021: 2 · 2020: 3 · 2019: 4 · 2018: 1 · 2017: 1
- FWCI
- 1.47
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Computers (publishedVersion, cc-by)
- Arrow@dit (Dublin Institute of Technology) (acceptedVersion)
- Cork Open Research Archive (University College Cork, Ireland) (acceptedVersion, cc-by)
- Cork Open Research Archive (University College Cork, Ireland) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Bundesministerium für Bildung und Forschung
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Social Sciences › Psychology › Developmental and Educational Psychology › Innovative Teaching and Learning Methods
- Social Sciences › Psychology › Developmental and Educational Psychology › Learning Styles and Cognitive Differences
- Keywords
- learner characteristics, need for cognition, work effort, computer self-efficacy, surface learning, learner confidence, e-module, test performance, online learning, personalized learning systems, testing effect
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.