Designing Activities to Teach Higher-Order Skills: How Feedback and Constraint Affect Learning of Experimental Design
Open accessAbstract
Active learning approaches to biology teaching, including simulation-based activities, are known to enhance student learning, especially of higher-order skills; nonetheless, there are still many open questions about what features of an activity promote optimal learning. Here we designed three versions of a simulation-based tutorial called Understanding Experimental Design that asks students to design experiments and collect data to test their hypotheses. The three versions vary the experimental design task along the axes of feedback and constraint, where constraint measures how much choice students have in performing a task. Using a variety of assessments, we ask whether each of those features affects student learning of experimental design. We find that feedback has a direct positive effect on learning. We further find that small changes in constraint have only subtle and mostly indirect effects on learning. This work suggests that designers of tools for teaching higher-order skills should strive to include feedback to increase impact and may feel freer to vary the degree of constraint within a range to optimize for other features such as the ability to provide immediate feedback and time-on-task.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2000) · doi:10.32614/r.manuals · Cited by 355,706 · see citing works
Hattie, J., & Timperley, H. (2007) · Review of Educational Research · doi:10.3102/003465430298487 · Cited by 12,905 · cited 3× · see citing works
“…at which the feedback is aimed (e.g., task vs. process vs. self-regulation), whether the feedback simply provides the correct answer, explains the rationale for that answer, or provides guidance for what the student should try next (Hattie and Timperley, 2007; Shute, 2008; Brooks et al. , 2019).”
Lakens, D. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00863 · Cited by 10,303 · see citing works
Active learning increases student performance in science, engineering, and mathematics
Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. P. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1319030111 · Cited by 9,787 · see citing works
(2013) · Cited by 6,708 · see citing works
Shute, V. J. (2008) · Review of Educational Research · doi:10.3102/0034654307313795 · Cited by 4,631 · see citing works
Practical Assessment, Research & Evaluation, 2000-2001.
Rudner, L. M., & Schafer, W. D. (2001) · Practical assessment, research & evaluation · Cited by 3,899 · see citing works
How Learning Works : Seven Research-Based Principles for Smart Teaching
Ambrose, S. A. (2010) · Cited by 2,342 · see citing works
The Power of Feedback Revisited: A Meta-Analysis of Educational Feedback Research
Wisniewski, B., Zierer, K., & Hattie, J. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.03087 · Cited by 1,199 · cited 3× · see citing works
“While our study was not designed to test different types of feedback, our effect size was large compared with many other studies of feedback effectiveness (Van der Kleij et al. , 2015; Wisniewski et al. , 2020).”
The learning effects of computer simulations in science education
Rutten, N., van Joolingen, W., & van der Veen, J. T. (2011) · Computers & Education · doi:10.1016/j.compedu.2011.07.017 · Cited by 1,079 · see citing works
Simulation-Based Learning in Higher Education: A Meta-Analysis
Chernikova, O., Heitzmann, N., Stadler, M., Holzberger, D., Seidel, T., & Fischer, F. (2020) · Review of Educational Research · doi:10.3102/0034654320933544 · Cited by 1,072 · see citing works
Paas, F. G. W. C., & Van Merriënboer, J. J. G. (1993) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/001872089303500412 · Cited by 921 · see citing works
Van der Kleij, F. M., Feskens, R. C. W., & Eggen, T. J. H. M. (2015) · Review of Educational Research · doi:10.3102/0034654314564881 · Cited by 714 · cited 8× · see citing works
“These results thus lend support to previous research indicating immediate, elaborated feedback that includes information on what to do next is effective (e.g., Brooks et al. , 2019; Wisniewski et al. , 2020), especially for higher-order tasks (Van der Kleij et al. , 2015).”
InfluentialFormative Assessment: A Meta-Analysis and a Call for Research
Kingston, N. M., & Nash, B. (2011) · Educational Measurement Issues and Practice · doi:10.1111/j.1745-3992.2011.00220.x · Cited by 443 · see citing works
Why Minimally Guided Teaching Techniques Do Not Work: A Reply to Commentaries
SWELLER, J., KIRSCHNER, P. A., & CLARK, R. E. (2007) · Educational Psychologist · doi:10.1080/00461520701263426 · Cited by 434 · see citing works
Blanchard, M. R., Southerland, S. A., Osborne, J. W., Sampson, V. D., Annetta, L. A., & Granger, E. M. (2010) · Science Education · doi:10.1002/sce.20390 · Cited by 426 · see citing works
Active Learning Not Associated with Student Learning in a Random Sample of College Biology Courses
Andrews, T. M., Leonard, M. J., Colgrove, C. A., & Kalinowski, S. T. (2011) · CBE—Life Sciences Education · doi:10.1187/cbe.11-07-0061 · Cited by 382 · see citing works
Evolution: Education and Outreach
(2013) · Evolution Education and Outreach · doi:10.1007/12052.1936-6434 · Cited by 375 · see citing works
Teaching Critical Thinking Skills In Higher Education: A Review Of The Literature
Behar‐Horenstein, L. S., & Niu, L. (2011) · Journal of College Teaching & Learning (TLC) · doi:10.19030/tlc.v8i2.3554 · Cited by 369 · see citing works
Just the Facts? Introductory Undergraduate Biology Courses Focus on Low-Level Cognitive Skills
Momsen, J. L., Long, T. M., Wyse, S. A., & Ebert‐May, D. (2010) · CBE—Life Sciences Education · doi:10.1187/cbe.10-01-0001 · Cited by 290 · see citing works
Zhu, M., Liu, O. L., & Lee, H.-S. (2020) · Computers & Education · doi:10.1016/j.compedu.2019.103668 · Cited by 229 · see citing works
Gobert, J. D., Sao Pedro, M., Raziuddin, J., & Baker, R. S. (2013) · Journal of the Learning Sciences · doi:10.1080/10508406.2013.837391 · Cited by 210 · see citing works
Reasoning About Natural Selection: Diagnosing Contextual Competency Using the ACORNS Instrument
Nehm, R. H., Beggrow, E. P., Opfer, J. E., & Ha, M. (2012) · The American Biology Teacher · doi:10.1525/abt.2012.74.2.6 · Cited by 177 · see citing works
Teaching the control-of-variables strategy: A meta-analysis
Schwichow, M., Croker, S. J., Zimmerman, C., Höffler, T. N., & Härtig, H. (2015) · Developmental Review · doi:10.1016/j.dr.2015.12.001 · Cited by 164 · see citing works
Online vs traditional homework: A systematic review on the benefits to students’ performance
Magalhães, P., Ferreira, D., da Cunha, J. S., & Rosário, P. J. S. L. F. (2020) · Computers & Education · doi:10.1016/j.compedu.2020.103869 · Cited by 159 · see citing works
Maier, U., Wolf, N., & Randler, C. (2016) · Computers & Education · doi:10.1016/j.compedu.2015.12.002 · Cited by 148 · see citing works
Theobald, R. J., & Freeman, S. (2014) · CBE—Life Sciences Education · doi:10.1187/cbe-13-07-0136 · Cited by 142 · see citing works
The Practice of Business Statistics
(2004) · Technometrics · doi:10.1198/tech.2004.s217 · Cited by 119 · see citing works
Elaborated feedback and learning: Examining cognitive and motivational influences
Wang, Z., Gong, S.-Y., Xu, S., & Hu, X.-E. (2019) · Computers & Education · doi:10.1016/j.compedu.2019.04.003 · Cited by 114 · see citing works
Instructional Utility and Learning Efficacy of Common Active Learning Strategies
McConnell, D. A., Chapman, L., Czajka, C. D., Jones, J. P., Ryker, K. D., & Wiggen, J. (2017) · Journal of Geoscience Education · doi:10.5408/17-249.1 · Cited by 113 · see citing works
Brooks, C., Carroll, A., Gillies, R. M., & Hattie, J. (2019) · The Australian journal of teacher education · doi:10.14221/ajte.2018v44n4.2 · Cited by 105 · see citing works
Dasgupta, A. P., Anderson, T. R., & Pelaez, N. J. (2014) · CBE—Life Sciences Education · doi:10.1187/cbe.13-09-0192 · Cited by 99 · see citing works
Arthurs, L. A., & Kreager, B. Z. (2017) · International Journal of Science Education · doi:10.1080/09500693.2017.1363925 · Cited by 99 · see citing works
Hubbard, J. K., Potts, M. A., & Couch, B. A. (2017) · CBE—Life Sciences Education · doi:10.1187/cbe.16-12-0339 · Cited by 93 · see citing works
Beggrow, E. P., Ha, M., Nehm, R. H., Pearl, D. K., & Boone, W. J. (2013) · Journal of Science Education and Technology · doi:10.1007/s10956-013-9461-9 · Cited by 75 · see citing works
Abraham, J. K., Meir, E., Perry, J., Herron, J. C., Maruca, S. L., & Stal, D. (2009) · Evolution Education and Outreach · doi:10.1007/s12052-009-0142-3 · Cited by 67 · see citing works
Self-citationHow Students Think about Experimental Design: Novel Conceptions Revealed by in-Class Activities
Brownell, S. E., Wenderoth, M. P., Theobald, R. J., Okoroafor, N., Koval, M., Freeman, S., Walcher-Chevillet, C. L., & Crowe, A. J. (2013) · BioScience · doi:10.1093/biosci/bit016 · Cited by 63 · see citing works
Randomization tests as alternative analysis methods for behavior‐analytic data
Craig, A. R., & Fisher, W. W. (2019) · Journal of the Experimental Analysis of Behavior · doi:10.1002/jeab.500 · Cited by 56 · see citing works
Development of the Biological Experimental Design Concept Inventory (BEDCI)
Deane, T., Nomme, K., Jeffery, E., Pollock, C. A., & Bírol, G. (2014) · CBE—Life Sciences Education · doi:10.1187/cbe.13-11-0218 · Cited by 56 · see citing works
Supporting middle school students’ science talk: A comparison of physical and virtual labs
Puntambekar, S., Gnesdilow, D., Tissenbaum, C. D., Narayanan, N. H., & Rebello, N. S. (2020) · Journal of Research in Science Teaching · doi:10.1002/tea.21664 · Cited by 56 · see citing works
Undergraduate students demonstrate common false scientific reasoning strategies
Woolley, J. S., Deal, A. M., Green, J., Hathenbruck, F., Kurtz, S. A., Park, T. K. H., Pollock, S. V., Transtrum, M. B., & Jensen, J. L. (2017) · Thinking Skills and Creativity · doi:10.1016/j.tsc.2017.12.004 · Cited by 54 · see citing works
Coordinating the effects of multiple variables: A skill fundamental to scientific thinking
Kuhn, D., Dreibelbis, M. A. P., & Wirkala, C. (2009) · Journal of Experimental Child Psychology · doi:10.1016/j.jecp.2009.01.009 · Cited by 49 · see citing works
Nehm, R. H. (2019) · Disciplinary and Interdisciplinary Science Education Research · doi:10.1186/s43031-019-0017-6 · Cited by 47 · see citing works
Abdullah, C., Parris, J., Lie, R., Guzdar, A., & Tour, E. (2015) · CBE—Life Sciences Education · doi:10.1187/cbe.14-10-0180 · Cited by 46 · see citing works
Agent-Based Modeling and Simulation in Mathematics and Biology Education
Bodine, E. N., Panoff, R. M., Voit, E. O., & Weisstein, A. E. (2020) · Bulletin of Mathematical Biology · doi:10.1007/s11538-020-00778-z · Cited by 41 · see citing works
Nehm, R. H., Finch, S. J., & Sbeglia, G. C. (2022) · BioScience · doi:10.1093/biosci/biac073 · Cited by 36 · see citing works
Uhl, J. D., Sripathi, K. N., Meir, E., MERRILL, J. E., Urban‐Lurain, M., & Haudek, K. C. (2021) · CBE—Life Sciences Education · doi:10.1187/cbe.20-06-0122 · Cited by 23 · see citing works
Meir, E., Wendel, D., Pope, D. S., Hsiao, L., Chen, D., & Kim, K. J. (2019) · Computers & Education · doi:10.1016/j.compedu.2019.103606 · Cited by 18 · see citing works
Shanks, R. A., Robertson, C. L., Haygood, C. S., Herdliksa, A. M., Herdliska, H. R., & Lloyd, S. A. (2017) · Journal of Microbiology and Biology Education · doi:10.1128/jmbe.v18i1.1194 · Cited by 13 · see citing works
Iterative design of a simulation-based module for teaching evolution by natural selection
Clarke‐Midura, J., Pope, D. S., Maruca, S. L., et al. (2018) · Evolution Education and Outreach · doi:10.1186/s12052-018-0078-6 · Cited by 11 · see citing works
Self-citationPelaez, N. J., Gardner, S. M., & Anderson, T. R. (2022) · Contributions from biology education research · doi:10.1007/978-3-030-98592-9_1 · Cited by 6 · see citing works
Meir, E. (2022) · Contributions from biology education research · doi:10.1007/978-3-030-98592-9_24 · Cited by 4 · see citing works
Flagged first, then by citations
Research works printed but not linked: 7
- [7]7. Baker, R. J. D. (2011). Gaming the system: A retrospective look. Philipp Comput J, 6(2011), 9– 13.
- [17]17. Colburn, A. (2000). An inquiry primer. Science Scope, 23(6), 42– 44.
- [22]22. Gobert, J. D. Sao Pedro, M. A. Baker, R. S. J.D. Toto, E. Montalvo, O. (2012). Leveraging educational data mining for real-time performance assessment of scientific inquiry skills within microworlds. Journal of Educational Data Mining, 4(1), 104– 143.
- [28]28. Klopfer, E. Scheintaub, H. Huang, W. Wendel, D. Roque, R. (2009). The simulation cycle: Combining games, simulations, engineering and science using StarLogo TNG. E-Learning, 6(1), 71– 96.
- [34]34. McMillan, J. H. Venable, J. C. Varier, D. (2013). Studies of the effect of formative assessment on student achievement: So much more is needed. Practical Assessment Research & Evaluation, 18(2), 1– 15.
- [49]49. Scalise, K. Gifford, B. (2006). Computer-based assessment in e-learning: A framework for constructing “intermediate constraint” questions and tasks for technology platforms. Journal of Technology, Learning, and Assessment, 4(6), 1– 46. http://files.eric.ed.gov/fulltext/EJ843857.pdf
- [57]57. Urry, L. A. Cain, M. L. Wasserman, S. A. Minorsky, P. V. Reece, J. B. (2017). Mastering Biology. New York, NY: Pearson Education Inc.
Cited by
Sorted by citations
Generative AI in Higher Education: Balancing Innovation and Integrity
Francis, N. J., Jones, S., & Smith, D. P. (2025) · British Journal of Biomedical Science · doi:10.3389/bjbs.2024.14048 · Cited by 295 · see citing works
“Assessment methods include open-ended questions [39], project-based assessments [40], and working-world problem-solving tasks [41] that encourage original thought and application of knowledge, thereby reducing the potential for GenAI misuse [42].”
Multichannel Wavelet Kernel Network for High Dimensional Inverse Modeling of Microwave Filters
Zhang, D., Zhou, M., Wang, Z., & Chen, H. (2024) · Electronics · doi:10.3390/electronics13234833 · Cited by 5 · see citing works
Advanced electromagnetic modeling method based on neuro-space mapping for SISL devices
Yan, S., Li, Z., Li, Y., Liu, W., & Wang, H. (2024) · IEICE Electronics Express · doi:10.1587/elex.21.20240289 · Cited by 2 · see citing works
Ill-Structured Design Challenges in First-Year Courses
Wilson-Fetrow, M., Mulchandani, A., Svihla, V., Lopez-Parra, R. D., Jobe, S. D., Eisenman, P., & Kapp, E. (2024) · ASEE Annual Conference & Exposition · doi:10.18260/1-2--47549 · Cited by 1 · see citing works
Embedding Generative AI as a digital capability into a year-long MSc skills programPreprint
Smith, D. P., Sokoya, D., Moore, S., Okonkwo, C., Boyd, C., Lacey, M. M., & Francis, N. (2024) · Research Square · doi:10.21203/rs.3.rs-5204546/v1 · Cited by 1 · see citing works
Pinto, J., Pedrosa, K. D. A., Pinto, R. M. F., Cavalcante, C. V. L., de Sousa, B. C. L., Magalhães, C. E. M., Farias, G. M. N., & da Silva, G. B. (2025) · Arquivos de Ciências da Saúde da UNIPAR · doi:10.25110/arqsaude.v29i3.2025-12139
Donkoh, J., Maruca, S. L., & Meir, E. (2025) · The American Biology Teacher · doi:10.1525/abt.2025.87.8.458
Chamely-Wiik, D. M., Meeroff, D. E., Frazier, E., Kwochka, W. R., Hansen, K., Aldarondo-Jeffries, M., Lancey, P., Nair, U., & Schneider, K. R. (2026) · Science Journal of Education · doi:10.11648/j.sjedu.20261404.11
Arıcı, F. (2026) · Journal of Science Education and Technology · doi:10.1007/s10956-026-10361-1
Engineering Communication Pedagogy in the Age of GenAI: A Taxonomy of Approaches
Bakhtawar, M., & Chong, A. (2026) · IEEE International Professional Communication Conference (ProComm) · doi:10.1109/procomm69759.2026.00048
- Authors
- Meir, E., Pope, D., Abraham, J. K., Kim, K. J., Maruca, S., & Palacio, J.
- Affiliations
- SimBiotic Software (United States) (US)
- University of Massachusetts Amherst (US)
- California State University, Fullerton (US)
- Harvard University (US)
- Corresponding author
- Meir, Eli
- Venue
- CBE—Life Sciences Education
- Published
- 2024-03
- Volume
- 23
- Issue
- 1
- Pages
- ar1-ar1
- ISSN
- 1931-7913 (ISSN-L)
- OpenAlex
- W4389636451
- Cited by
- 10
- References
- 49
- Citations by year
- 2026: 3 · 2025: 3 · 2024: 4
- FWCI
- 7.40
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by-nc-sa
- Locations
- CBE—Life Sciences Education (publishedVersion, cc-by-nc-sa)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Education › Innovative Teaching Methods
- Social Sciences › Education › Science Education and Pedagogy
- Physical Sciences › Mathematics › Statistics and Probability › Statistics Education and Methodologies
- Keywords
- experimental design, feedback, simulation-based training, active learning, biology education, higher-order thinking skills, tutorial design, hypothesis testing, student learning
- MeSH terms
- Feedback, Humans, Surveys and Questionnaires, Research Design, Students, Problem-Based Learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
