The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Subgoals, Context, and Worked Examples in Learning Computing Problem Solving
Open accessAbstract
Recent empirical results suggest that the instructional material used to teach computing may actually overload students' cognitive abilities. Better designed materials may enhance learning by reducing unnecessary load. Subgoal labels have been shown to be effective at reducing the cognitive load during problem solving in both mathematics and science. Until now, subgoal labels have been given to students to learn passively. We report on a study to determine if giving learners subgoal labels is more or less effective than asking learners to generate subgoal labels within an introductory CS programming task. The answers are mixed and depend on other features of the instructional materials. We found that student performance gains did not replicate as expected in the introductory CS task for those who were given subgoal labels. Computer science may require different kinds of problem-solving or may generate different cognitive demands than mathematics or science.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“A review of self-explanation shows that along with WEs, self-explanation prompts help students foster greater understanding [16] and improves students' problem solving performance [12].”
From Exploring the Impact of Worked Examples in a Novice Programming Environment (2019) · cites this paper as [12]
In the citing paper
“Morrison et al. showed that students who self-explained subgoal labels perform better than those who were not given subgoal labels [12].”
From Exploring the Impact of Worked Examples in a Novice Programming Environment (2019) · cites this paper as [12]
Citations
Cites
(1999) · Cited by 12,812
Cognitive Architecture and Instructional Design
Sweller, J., van Merrienboer, J. J. G., & Paas, F. G. W. C. (1998) · Educational Psychology Review · doi:10.1023/a:1022193728205 · Cited by 6,023 · cited 3×
“It is believed that worked examples, when carefully designed, can accomplish both of these goals [24].”
BackgroundSweller, J., Ayres, P., & Kalyuga, S. (2011) · doi:10.1007/978-1-4419-8126-4 · Cited by 5,531
Sweller, J. (2012) · Encyclopedia of the Sciences of Learning · doi:10.1007/978-1-4419-1428-6_446 · Cited by 4,405
Cognitive Load Theory and Complex Learning: Recent Developments and Future Directions
van Merriënboer, J. J. G., & Sweller, J. (2005) · Educational Psychology Review · doi:10.1007/s10648-005-3951-0 · Cited by 2,206 · cited 2×
“The central problem identified by Cognitive Load Theory (CLT) is that learning is impaired when the total amount of processing requirements exceeds the limited capacity of working memory [20].”
BackgroundElement Interactivity and Intrinsic, Extraneous, and Germane Cognitive Load
Sweller, J. (2010) · Educational Psychology Review · doi:10.1007/s10648-010-9128-5 · Cited by 2,020 · cited 4×
“Worked examples introduce extraneous cognitive load because they are necessarily specific to a context, and students must process the incidental information about the context even though it is not relevant to the underlying procedure [26].”
BackgroundInfluentialSelf‐Explanations: How Students Study and Use Examples in Learning to Solve Problems
Chi, M. T. H., Bassok, M., Lewis, M. W., Reimann, P., & Glaser, R. (1989) · Cognitive Science · doi:10.1207/s15516709cog1302_1 · Cited by 1,714 · cited 2×
“When studying examples, learners tend to focus on incidental features rather than the fundamental features because incidental features are easier to grasp and novices do not have the necessary domain knowledge to recognize fundamental features of examples [11].”
BackgroundDevelopment of an instrument for measuring different types of cognitive load
Leppink, J., Paas, F., Van der Vleuten, C. P. M., Van Gog, T., & Van Merriënboer, J. J. G. (2013) · Behavior Research Methods · doi:10.3758/s13428-013-0334-1 · Cited by 1,066 · cited 2×
“Extraneous load is the load placed on working memory that does not contribute directly toward the learning of the material---for example, the resources consumed while understanding poorly written text or diagrams without sufficient clarity [13].”
BackgroundLearning from Examples: Instructional Principles from the Worked Examples Research
Atkinson, R. K., Derry, S. J., Renkl, A., & Wortham, D. (2000) · Review of Educational Research · doi:10.2307/1170661 · Cited by 1,029 · cited 3×
“Worked examples are most effective when used in worked example-practice pairs [2].”
BackgroundMethodInfluential(2010) · Cambridge University Press eBooks · doi:10.1017/cbo9780511844744 · Cited by 575 · cited 1×
“The central problem identified by Cognitive Load Theory (CLT) is that learning is impaired when the total amount of processing requirements exceeds the limited capacity of working memory [20].”
BackgroundRenkl, A., & Atkinson, R. K. (2003) · Educational Psychologist · doi:10.1207/s15326985ep3801_3 · Cited by 540 · cited 1×
“Categories and Subject Descriptors K.3.2 [Computers and Education]: Computer and Information Science Education: computer science education, information systems education General Terms Measurement, Design, Experimentation Keywords Subgoal labels; Cognitive Load; Contextual Transfer”
BackgroundInfluentialEffects of pairs of problems and examples on task performance and different types of cognitive load
Leppink, J., Paas, F., van Gog, T., van der Vleuten, C. P. M., & van Merriënboer, J. J. G. (2014) · Learning and Instruction · doi:10.1016/j.learninstruc.2013.12.001 · Cited by 520 · cited 1×
“There are several ways to reduce cognitive load, including using worked examples [14].”
Optimizing learning from examples using animated pedagogical agents.
Atkinson, R. K. (2002) · Journal of Educational Psychology · doi:10.1037//0022-0663.94.2.416 · Cited by 472 · cited 1×
“Studies [3, 4, 8–10, 15, 16] have consistently found that subgoal-oriented instructions improved problem solving performance across a variety of STEM domains, such as programming (e.g., [15]) and statistics (e.g., [10]).”
ResultThe subgoal learning model: Creating better examples so that students can solve novel problems.
Catrambone, R. (1998) · Journal of Experimental Psychology: General · doi:10.1037//0096-3445.127.4.355 · Cited by 327 · cited 3×
“Subgoal labels are names given to a set of steps in the solution process allowing the user to “chunk” the information to ease learning [10].”
BackgroundMethodEiriksdottir, E., & Catrambone, R. (2011) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720811419154 · Cited by 174
The FCS1: a language independent assessment of CS1 knowledge
Tew, A. E., & Guzdial, M. (2011) · Proceedings of the 42nd ACM technical symposium on Computer science education · doi:10.1145/1953163.1953200 · Cited by 170 · cited 1×
“Java Computer Science, Information Technology, Software Engineering, NonMajors (mostly physics and math) Pseudo-code is relatively easy for programmers to understand regardless of the programming languages that they know [27].”
BackgroundSelf-citationGeneralizing solution procedures learned from examples.
Catrambone, R. (1996) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037/0278-7393.22.4.1020 · Cited by 154
Margulieux, L. E., Guzdial, M., & Catrambone, R. (2012) · Proceedings of the ninth annual international conference on International computing education research · doi:10.1145/2361276.2361291 · Cited by 149 · cited 2×
“While these cognitive load reducing techniques have been empirically tested in math and science disciplines, we have been the first to test these with computer science learning [15].”
BackgroundSelf-citationMeasuring cognitive load in introductory CS: adaptation of an instrument
Morrison, B. B., Dorn, B., & Guzdial, M. (2014) · Proceedings of the tenth annual conference on International computing education research · doi:10.1145/2632320.2632348 · Cited by 147 · cited 2×
“Having completed the instructional period, participants were then asked to complete a 10 item survey designed to measure cognitive load [18].”
MethodSelf-citationAiding subgoal learning: Effects on transfer.
Catrambone, R. (1995) · Journal of Educational Psychology · doi:10.1037/0022-0663.87.1.5 · Cited by 144 · cited 1×
“Subgoal labels that are independent from a specific context have been the most effective type of subgoal labels in the past [7, 10].”
BackgroundImproving examples to improve transfer to novel problems
Catrambone, R. (1994) · Memory & Cognition · doi:10.3758/bf03198399 · Cited by 129 · cited 1×
“Subgoal labeling is a technique used to promote subgoal learning that has been used to help learners recognize the fundamental structure of the procedure being exemplified in worked examples [8–10].”
MethodAnimated Demonstrations for Learning Procedural Computer-Based Tasks
Palmiter, S. L., & Elkerton, J. (1993) · Human-Computer Interaction · doi:10.1207/s15327051hci0803_1 · Cited by 119 · cited 1×
“reducing illusions of understanding caused by shallow processing thus inducing deeper processing of information [5, 12, 19].”
BackgroundLearning From Examples: Fostering Self-Explanations in Computer-Based Learning Environments
Renkl, A., & Atkinson, R. K. (2002) · Interactive Learning Environments · doi:10.1076/ilee.10.2.105.7441 · Cited by 74 · cited 2×
“hinder learning but is inherent in worked examples [21].”
BackgroundAtkinson, R. K., Catrambone, R., & Merrill, M. M. (2003) · Journal of Educational Psychology · doi:10.1037/0022-0663.95.4.762 · Cited by 66 · cited 2×
“Subgoal labels are believed to be effective because they visually group the steps of worked examples into subgoals and meaningfully label those groups [1].”
BackgroundSegmentation of Worked Examples: Effects on Cognitive Load and Learning
Spanjers, I. A. E., van Gog, T., & van Merriënboer, J. J. G. (2011) · Applied Cognitive Psychology · doi:10.1002/acp.1832 · Cited by 62 · cited 1×
“Segmenting includes separating portions of the worked example to isolate each step in the process [23].”
Backgroundvan Gog, T., & PAAS, F. G. W. C. (2012) · Springer eBooks · doi:10.1007/978-1-4419-1428-6_412 · Cited by 55
Atkinson, R. K., & Derry, S. J. (2000) · Cited by 22
Improving problem solving performance in computer-based learning environments through subgoal labels
Margulieux, L. E., & Catrambone, R. (2014) · ACM Conference on Learning @ Scale (L@S) · doi:10.1145/2556325.2567853 · Cited by 17
Self-citation
Research works printed but not linked: 3
- 10.1037/0022-0663.94.2.416 · doi:10.1037/0022-0663.94.2.416 · Paper page
- Bjork R. A., Memory and metamemory considerations in the training of human beings. Metacognition: Knowing about Knowing
- 10.1037/0096-3445.127.4.355 · doi:10.1037/0096-3445.127.4.355 · Paper page
Cited by
Prather, J., Reeves, B. N., Denny, P., Becker, B. A., Leinonen, J., Luxton-Reilly, A., Powell, G., Finnie-Ansley, J., & Santos, E. A. (2024) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3617367 · Cited by 256
“Another form of scaffolding could be Copilot generating only comments outlining the program’s general structure, similar to subgoal labels [76].”
BackgroundLearner-Centered Design of Computing Education: Research on Computing for Everyone
Guzdial, M. (2015) · Synthesis lectures on human-centered informatics · doi:10.2200/s00684ed1v01y201511hci033 · Cited by 244
Self-citationReplication, Validation, and Use of a Language Independent CS1 Knowledge Assessment
Parker, M. C., Guzdial, M., & Engleman, S. (2016) · Proceedings of the 2016 ACM Conference on International Computing Education Research · doi:10.1145/2960310.2960316 · Cited by 184
“The replication process is the same as in any education domain [3] but we know that education findings do not always map directly from one domain to another, especially in computing [23].”
BackgroundSelf-citationNovice Programmers and Introductory Programming
Robins, A. V. (2019) · The Cambridge Handbook of Computing Education Research · doi:10.1017/9781108654555.013 · Cited by 175
The Role of Self-Regulation in Programming Problem Solving Process and Success
Loksa, D., & Ko, A. J. (2016) · Proceedings of the 2016 ACM Conference on International Computing Education Research · doi:10.1145/2960310.2960334 · Cited by 143
“For example, recent studies have explored worked examples and the effect of sub-goal labels, finding that examples and sub-goal labels can promote greater problem solving success [20,22,23].”
MethodResult
Metadata
- Authors
- Morrison, B. B., Margulieux, L. E., & Guzdial, M.
- Affiliations
- Georgia Institute of Technology (US)
- Venue
- Proceedings of the eleventh annual International Conference on International Computing Education Research
- Publisher
- ACM
- Published
- 2015-08-09
- Pages
- 21-29
- OpenAlex
- W2041781135
- Cited by
- 174
- References
- 28
- Citations by year
- 2026: 10 · 2025: 8 · 2024: 17 · 2023: 9 · 2022: 13 · 2021: 19 · 2020: 14 · 2019: 53 · 2018: 9 · 2017: 10 · 2016: 10 · 2015: 2
- FWCI
- 7.39
- Type
- conference paper
- Language
- en
- Open access
- Locations
- ACM Conference on International Computing Education Research (ICER) (publishedVersion)
- DigitalCommons@UNO (submittedVersion)
- Digital Archive @ GSU (submittedVersion)
- Funders
- National Science Foundation (1138378)
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Visual and Cognitive Learning Processes
- Social Sciences › Psychology › Developmental and Educational Psychology › Innovative Teaching and Learning Methods
- Keywords
- worked examples, cognitive load, programming education, introductory computer science, problem solving, subgoal generation, instructional design
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.