ManyClasses 2: The effects of prequestions on media interactions and learning.
Open accessPublished version of a preprint
The DOI matches its Crossref record; 101 of the 101 references it cites were found in a registry or an index; 8 of the 34 sources its claims cite were checked for support.
Abstract
We conducted a ManyClasses study in 30 different classes, where student participants (n = 1,571) were assigned to watch two online lecture videos, and one of these videos (order randomized) was immediately preceded by ungraded multiple-choice prequestions, with no feedback or answers shown. In the laboratory, decades of research have observed that prequestions reliably improve learning from subsequent instruction when compared to instruction not preceded by prequestions, but evidence in authentic education settings has been elusive. The current experiment, embedded in authentic classes ranging from sixth grade through senior-level college, with class-specific videos and questions created by teachers, confirmed that prequestions improve average student learning performance, measured by delayed classroom assessments that included the prequestions. Furthermore, we examined how prequestions affected students’ interactions with the online videos. Despite the generalizable average benefit of prequestions for student learning, we found that prequestions also caused some students to disengage, skipping the assigned video entirely. Among students who did watch the videos, there was no increase in the amount viewed after answering prequestions compared to videos with no prequestions. These findings lead us to suggest that prequestions do not increase attention to the subsequent learning materials, but rather, prequestions cause learners to activate a relevant mental model in advance of…
Overview
CoChat Summary
TL;DR
In a ManyClasses experiment across 30 authentic classes, ungraded multiple-choice prequestions before short online videos improved average delayed assessment performance, but also reduced video engagement for some students. The authors argue the benefits are better explained by activation of a relevant mental model than by increased attention to instruction.
Key findings
- Prequestions improved delayed learning: students scored 73.7% on prequestioned items versus 68.8% on control items, with a modal log-odds effect of 0.24, 95% HDI [0.082, 0.42], or an odds ratio of 1.28. Directionally better performance occurred in 25 of 30 classes.
- Prequestions reduced media engagement: students watched 64.5% of videos preceded by prequestions versus 69.2% of control videos, estimated difference mode = 4.9%, 95% HDI [−8.1%, −1.9%]. Non-initiation of playback was 21.6% after prequestions versus 16.5% without prequestions, log odds = −0.42, 95% HDI [−0.70, −0.14].
- Prequestion accuracy moderated learning benefits: the interaction between prequestion condition and individual prequestion score was log odds = 0.80, 95% HDI [0.45, 1.13]. Higher class-level delayed-assessment performance also moderated larger prequestion benefits, interaction log odds = 0.74, 95% HDI [0.14, 1.37].
- Among students who watched the videos, there was no credible interaction between prequestions and percentage of video viewed on delayed assessment scores, log odds = 0.044, 95% HDI [−0.21, 0.30], nor with initiating playback, log odds = 0.15, 95% HDI [−0.13, 0.44].
Methods
- Within-subjects crossover experiment embedded in 30 classes (27 teachers; 7 campuses/institutions; 70% college, 30% precollege; grade levels 6–16). Students were randomized to see prequestions before one of two short online lecture videos and no prequestions before the other, with order randomized.
- Final analytic sample was 1,571 students from 3,507 enrolled; 1,757 consented/assented, and 186 were excluded for missing delayed outcome scores. Teachers created two videos (≤5 min each) and at least four multiple-choice prequestions per video; the same items appeared unmodified on a later graded assessment, with mean delay 11.7 days (SD = 9.7).
- Data were analyzed with Bayesian hierarchical mixed-effects models using brms, estimating condition effects overall and by class, with random intercepts for participants nested within classes and a class-level random effect of exposure order. Moderator analyses were exploratory.
Limitations
- Stated: the study did not collect student demographic data, which the authors regret because it would have allowed more comprehensive description and disaggregated analyses.
- Stated: the XPRIZE competition timeline and sample requirements limited recruitment and flexibility; the implementation fixed key features (online assignments, multiple-choice prequestions, short videos), so findings should not be interpreted as evaluating diverse ways of implementing prequestions.
- Stated: class-level estimates were uncertain, the study was underpowered for two-way interactions between class-level properties and student prequestion accuracy, and the authors cannot claim strong evidence of effectiveness in any particular class. The study also did not assess whether prequestion effects are general or specific to prequestioned concepts.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
6 Valid · 2 Not valid · 14 Not checked · 124 Own results and interpretations
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“On delayed assessments, students performed no better on items that had been introduced as prequestions (and then again as postquestions) when compared with items that had only been presented as postquestions at the end of lectures.”
What the sources say
- Not checked — no full text of the sourcecites Carpenter, Rahman, & Perkins, 2018 source
- Supportedcites Geller et al., 2017 source
The Methods describe questions asked both before and after class and questions asked only after class, followed by a weekly quiz. The Results (‘The effect of prequestions on retrieval practice’; page unavailable) report no significant difference between those question types on the delayed weekly quizzes (p = 0.44). (LLM, conf=0.92)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“In the two experiments where no prequestion effect was observed, the classes also included additional interactive elements (team-based learning activities in Geller et al., 2017; and postquestions in Carpenter, Rahman, & Perkins, 2018), and this has led researchers to reasonably assume that the effect of prequestions may be muf fl ed by other concomitant learning activities.”
What the sources say
- Supportedcites Geller et al., 2017 source
The Results report no significant advantage for prequestioned over non-prequestioned information. The Methods (Procedure; page not provided) say that the same lessons included team-based learning activities. This supports the part of the sentence attributed to Geller et al.; the other experiment and the proposed explanation are not attributed to this source. (LLM, conf=0.94)
- Not checked — no full text of the sourcecites Carpenter, Rahman, & Perkins, 2018 source
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“but studies conducted in authentic education environments have produced scant and inconsistent evidence of such bene fi ts.”
What the sources say
- Not checked — no full text of the sourcecites Soderstrom & Bjork, 2023 source
- Not checked — no full text of the sourcecites Carpenter, Rahman, & Perkins, 2018 source
- Supportedcites Geller et al., 2017 source
The Background describes classroom evidence as largely unexplored and reports mixed results from the one earlier known classroom study. The Discussion reports no benefit in the current classroom study and concludes that prequestions do not produce consistent, reliable effects in classroom environments. (LLM, conf=0.94)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Controlled laboratory studies have produced abundant evidence of the bene fi ts of prequestions for learning and memory (Pan & Carpenter, 2023; St. Hilaire et al., 2024), but studies conducted in authentic education environments have produced scant and inconsistent evidence of such bene fi ts.”
What the sources say
- Supportedcites Pan & Carpenter, 2023 source
The review’s Abstract reports a growing body of research showing that prequestioning and pretesting can benefit learning and memory. It describes robust experimental studies reporting learning benefits and notes that more than 60 peer-reviewed articles have since been published on the topic. The review cautions that the size of the benefit varies by procedure and assessment. (LLM, conf=0.90)
- Not matchedcites St. Hilaire et al., 2024
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“the estimated interaction between the prequestion effect and the number of other assessments in the course is very close to zero (interaction estimate in log odds = -0.005, 95% HDI = [ -0.56, 0.52]).”
What the sources say
- Could not verifycites Szpunar et al., 2013 sourceExperimental — accuracy not yet measured
The Materials and Methods describe tests inserted between segments of a single video lecture, not prequestions or the number of other assessments in a course. The Results report a nonsignificant interaction concerning mind wandering and comprehension, but not the claimed prequestion interaction or its log-odds estimate and 95% HDI. Searches for those terms and values found no supporting passage. Page numbers are unavailable in our copy. (LLM, conf=0.78)
- Could not verifycites Yang et al., 2018 sourceExperimental — accuracy not yet measured
The review’s Summary and Practical implications for instructors discuss interim testing and classroom assessments, but do not report an interaction between prequestioning and the number of other course assessments. Searches for that interaction, its log-odds estimate, and its HDI found no matching result in our copy. (LLM, conf=0.75)
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
56 more claims
Create a free account to see all 146 claims and every source verdict.
- It may state two separate claims as one. 71
- Probably not a substantial claim. 22
- The quote only partly states this claim. 25
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“| | 49 | Lifespan development | 14.5 | 31 | 5 (19%) | 55 |”
Reported 19 · recomputed 16.129032
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ A.
“(course ids = 37, 47, 48, and 49)”
§ Study Context
“the fi rst wave was an initial pilot (course id = 38)”
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 4
- Chi, M. T. H. (2000). Self-explaining: The dual processes of generating inference and repairing mental models. In R. Glaser (Ed.), Advances in instructional psychology: Educational design and cognitive science (Vol. 5, pp. 161 -238). Lawrence Erlbaum Associates Publishers.
- Duchastel, P. C. (1983). Interpreting adjunct question research: Processes and ecological validity. Human Learning: Journal of Practical Research & Applications , 2 (1), 1 -5.
- Kruschke, J. K. (2014). Doing Bayesian data analysis: A tutorial with R, JAGS, and Stan (2nd ed.). Academic Press.
- Plummer, M. (2003). JAGS: A program for analysis of Bayesian graphical models using Gibbs sampling . Proceedings of the 3rd international workshop on distributed statistical computing (DSC 2003).
Cited by
A field-initiated vision of research infrastructure for STEM education
Motz, B. A., Diekman, A. B., Goldstone, R. L., Green, D. J.-A., Fyfe, E. R., Bernardini, S., Emerson, K. T. U., Jankowski, H. E., Ansari, S., Arney, M., et al. (2025) · International Journal of STEM Education · doi:10.1186/s40594-025-00581-z · Cited by 1
“Across this diverse range of local studies, the incubator sees a need to understand the effectiveness, generalizability, and heterogeneity of instructional interventions at a high-level (Bryan et al., 2021; de Leeuw et al., 2022; Motz et al., 2025).”
Self-citation
Metadata
- Authors
- Motz, B. A., Chinni, A., de Leeuw, J. R., Jankowski, H. E., Aggarwal, A., Amato, M., Berlin, K., Britten, K., Brown, A., Cerchiaro, M. L., Evans, N. J., Findley, A., Gorman, R., Gregg, K. D., Hansen, K., Hollender, H., Hullinger, R. A., Larkin, P., Lion, M., Long, R. E., Mannarino, A. J., Mocko, M. E., Moore, K., Packowski, J. A., Palmer, C., Ritchie, K. L., Scott, J., Stanton, M., Talcott, L., Wagner, L. N., Waite, M., Yeager, R. G., & Fyfe, E. R.
- ORCID
- Venue
- Journal of Educational Psychology
- Published
- 2026-02
- Volume
- 118
- Issue
- 2
- Pages
- 153-175
- ISSN
- 0022-0663 (ISSN-L)
- 1939-2176
- OpenAlex
- W4413776691
- Cited by
- 1
- References
- 0
- FWCI
- 1.05
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Journal of Educational Psychology (publishedVersion, cc-by-nc-nd)
- IUScholarWorks (Indiana University) (submittedVersion, cc-by-nc-nd)
- Funders
- US Department of Education (R305N210035)
- XPRIZE Digital Learning Challenge
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Visual and Cognitive Learning Processes
- Social Sciences › Education › Innovative Teaching Methods
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory Processes and Influences
- Keywords
- prior knowledge activation, student engagement, learning from instruction, rich-get-richer effect, mental models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-28, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-28 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: ManyClasses 2: The effects of prequestions on media interactions and learning. https://app.cochat.ai/papers/10.1037/edu0000978
- The paper
- Motz, B. A., Chinni, A., de Leeuw, J. R., Jankowski, H. E., Aggarwal, A., Amato, M., Berlin, K., Britten, K., Brown, A., Cerchiaro, M. L., Evans, N. J., Findley, A., Gorman, R., Gregg, K. D., Hansen, K., Hollender, H., Hullinger, R. A., Larkin, P., Lion, M., . . . Fyfe, E. R. (2026). ManyClasses 2: The effects of prequestions on media interactions and learning. Journal of Educational Psychology. https://doi.org/10.1037/edu0000978
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.