The DOI matches its Crossref record; 49 of the 51 references it cites were found in a registry or an index.
Rubrics to assess critical thinking and information processing in undergraduate STEM courses
Open accessAbstract
Background: Process skills such as critical thinking and information processing are commonly stated outcomes for STEM undergraduate degree programs, but instructors often do not explicitly assess these skills in their courses. Students are more likely to develop these crucial skills if there is constructive alignment between an instructor’s intended learning outcomes, the tasks that the instructor and students perform, and the assessment tools that the instructor uses. Rubrics for each process skill can enhance this alignment by creating a shared understanding of process skills between instructors and students. Rubrics can also enable instructors to reflect on their teaching practices with regard to developing their students’ process skills and facilitating feedback to students to identify areas for improvement. Results: Here, we provide rubrics that can be used to assess critical thinking and information processing in STEM undergraduate classrooms and to provide students with formative feedback. As part of the Enhancing Learning by Improving Process Skills in STEM (ELIPSS) Project, rubrics were developed to assess these two skills in STEM undergraduate students’ written work. The rubrics were implemented in multiple STEM disciplines, class sizes, course levels, and institution types to ensure they were practical for everyday classroom use. Instructors reported via surveys that the rubrics supported assessment of students’ written work in multiple STEM learning environments. Graduate teaching assistants also indicated that they could effectively use the rubrics to assess student work and that the rubrics clarified the instructor’s expectations for how they should assess students. Students reported that they understood the content of the rubrics and could use the feedback provided by the rubric to change their future performance. Conclusion: The ELIPSS rubrics allowed instructors to explicitly assess the critical thinking and information processing skills that they wanted their students to develop in their courses. The instructors were able to clarify their expectations for both their teaching assistants and students and provide consistent feedback to students about their performance. Supporting the adoption of active-learning pedagogies should also include changes to assessment strategies to measure the skills that are developed as students engage in more meaningful learning experiences. Tools such as the ELIPSS rubrics provide a resource for instructors to better align assessments with intended learning outcomes.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“there are separate instruments focusing specifically on STEM students’ critical thinking (e.g., White et al., 2011), information processing (Reynders et al., 2020), scientific reasoning (e.g., Moore and Rubbo, 2012), and motivation and self-efficacy when reading PSL (Chatzikyriakidou and McCartney, 2022).”
From Reading Primary Scientific Literature: Approaches for Teaching Students in the Undergraduate STEM Classroom (2023) · cites this paper as Reynders et al., 2020
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Becoming a Self-Regulated Learner: An Overview
Zimmerman, B. J. (2002) · Theory Into Practice · doi:10.1207/s15430421tip4102_2 · Cited by 10,197 · cited 2×
“Zimmerman’s theory of selfregulated learning (Zimmerman, 2002) provides a rationale for tailoring assessments to provide feedback to both students and instructors.”
BackgroundMethodActive learning increases student performance in science, engineering, and mathematics
Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. P. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1319030111 · Cited by 9,787
Assessment and Classroom Learning
Black, P., & Wiliam, D. (1998) · Assessment in Education: Principles, Policy & Practice · doi:10.1080/0969595980050102 · Cited by 8,152 · cited 1×
“…need consistent feedback from an instructor and/or self-assessment throughout a course to provide a benchmark for their current performance and identify what they can do to improve their process skills (Black & Wiliam, 1998; Butler & Winne, 1995; Hattie & Gan, 2011; Nicol & Macfarlane-Dick, 2006).”
BackgroundNicol, D. J., & Macfarlane‐Dick, D. (2006) · Studies in Higher Education · doi:10.1080/03075070600572090 · Cited by 5,846
Enhancing teaching through constructive alignment
Biggs, J. (1996) · Higher Education · doi:10.1007/bf00138871 · Cited by 4,084
Feedback and Self-Regulated Learning: A Theoretical Synthesis
Butler, D. L., & Winne, P. H. (1995) · Review of Educational Research · doi:10.3102/00346543065003245 · Cited by 3,334
Education for Life and Work: Developing Transferable Knowledge and Skills in the 21st Century
Pellegrino, J. W., & Hilton, M. L. (2012) · Cited by 2,038
(1993) · Cited by 1,939
The use of scoring rubrics: Reliability, validity and educational consequences
Jonsson, A., & Svingby, G. (2007) · Educational Research Review · doi:10.1016/j.edurev.2007.05.002 · Cited by 1,509 · cited 3×
“Jonsson and Svingby (2007) found that most of the rubrics they reviewed had adjacent agreement scores of 90% or greater.”
BackgroundMethodInfluentialPsychology of learning for instruction
Lohr, L., & Chang, S. (2005) · Educational Technology Research and Development · doi:10.1007/bf02504860 · Cited by 1,165
Constructivism: A theory of knowledge
Bodner, G. M. (1986) · Journal of Chemical Education · doi:10.1021/ed063p873 · Cited by 1,027
Singer, S. R., & Smith, K. A. (2013) · Journal of Engineering Education · doi:10.1002/jee.20030 · Cited by 852
The use of scoring rubrics for formative assessment purposes revisited: A review
Panadero, E., & Jonsson, A. (2013) · Educational Research Review · doi:10.1016/j.edurev.2013.01.002 · Cited by 839
Defining higher order thinking
Lewis, A., & Smith, D. (1993) · Theory Into Practice · doi:10.1080/00405849309543588 · Cited by 772
Purposely Teaching for the Promotion of Higher-order Thinking Skills: A Case of Critical Thinking
Barak, M., Ben‐Chaim, D., & Uri, Z. (2007) · Research in Science Education · doi:10.1007/s11165-006-9029-2 · Cited by 757
Why some students don't learn chemistry: Chemical misconceptions
Nakhleh, M. B. (1992) · Journal of Chemical Education · doi:10.1021/ed069p191 · Cited by 726
Assessment in Education: Principles, Policy & Practice
Wiliam, D. (2008) · Cited by 696
Critical Thinking: A Literature Review
Lai, E. (2011) · Cited by 693
Scoring Rubric Development: Validity and Reliability
Moskal, B., & Leydens, J. A. (2020) · Scholarworks (University of Massachusetts Amherst) · doi:10.7275/q7rm-gg74 · Cited by 502
Constructive alignment in university teaching
Kandlbinder, P. (2014) · Cited by 479
Critical Thinking and Science Education
Bailin, S. (2002) · Science & Education · doi:10.1023/a:1016042608621 · Cited by 404
Scoring Rubrics: What, When and How?
Moskal, B. (2020) · ScholarWorks@UMassAmherst (University of Massachusetts Amherst) · doi:10.7275/a5vq-7q66 · Cited by 333
Process Oriented Guided Inquiry Learning (POGIL)
Moog, R. S., & Spencer, J. N. (2008) · ACS symposium series · doi:10.1021/bk-2008-0994 · Cited by 294
Assessing 21st Century Skills: Summary of a Workshop
Koenig, J. A. (2011) · ATE Central (National Science Foundation) · Cited by 279
The quality and effectiveness of descriptive rubrics
Brookhart, S., & Chen, F. (2015) · Educational Review · doi:10.1080/00131911.2014.929565 · Cited by 254
Schmidt, H. G., de Volder, M. L., de Grave, W. S., Moust, J. H. C., & Patel, V. L. (1989) · Journal of Educational Psychology · doi:10.1037/0022-0663.81.4.610 · Cited by 214
Critical thinking and the disciplines reconsidered
Davies, M. (2013) · Higher Education Research & Development · doi:10.1080/07294360.2012.697878 · Cited by 205
An Information Processing Analysis of Expert and Novice Teachers' Problem Solving
Swanson, H. L., O’Connor, J. E., & Cooney, J. B. (1990) · American Educational Research Journal · doi:10.3102/00028312027003533 · Cited by 186
Hattie, J., & Gan, M. J. S. (2011) · Routledge eBooks · doi:10.4324/9780203839089-22 · Cited by 172
The Extent to Which Critical Thinking Is Subject-Specific: Further Clarification
Ennis, R. H. (1990) · Educational Researcher · doi:10.3102/0013189x019004013 · Cited by 118
Gafney, L., & Varma‐Nelson, P. (2008) · Springer eBooks · doi:10.1007/978-1-4020-6186-8 · Cited by 109
The effect of the biology critical thinking project on the development of critical thinking
Zohar, A. E., Weinberger, Y., & Tamir, P. (1994) · Journal of Research in Science Teaching · doi:10.1002/tea.3660310208 · Cited by 107
What stands and develops between creative and critical thinking? Argumentation?
Glassner, A., & Schwarz, B. B. (2006) · Thinking Skills and Creativity · doi:10.1016/j.tsc.2006.10.001 · Cited by 96
Rau, M. A., Kennedy, K. J., Oxtoby, L. J., Bollom, M. A., & Moore, J. W. (2017) · Journal of Chemical Education · doi:10.1021/acs.jchemed.7b00240 · Cited by 88
Facione, P. A. (1990) · Cited by 82
Developing Student Process Skills in a General Chemistry Laboratory
Reynders, G. J., Suh, E. H., Cole, R. S., & Sansom, R. L. (2019) · Journal of Chemical Education · doi:10.1021/acs.jchemed.9b00441 · Cited by 76
Self-citationSaxton, E. A., Belanger, S., & Becker, W. (2012) · Assessing Writing · doi:10.1016/j.asw.2012.07.002 · Cited by 75
Danczak, S., Thompson, C. D., & Overton, T. (2017) · Chemistry Education Research and Practice · doi:10.1039/c6rp00249h · Cited by 74
Simonson, S. R. (2019) · ScholarWorks (Boise State University) · Cited by 71
Smit, R., & Birri, T. (2014) · Studies In Educational Evaluation · doi:10.1016/j.stueduc.2014.02.002 · Cited by 49
Toward a Theory of Critical Thinking.
Facione, P. A. (1984) · Liberal education · Cited by 48
Kumi, B. C., Olimpo, J. T., Bartlett, F., & Dixon, B. (2013) · Chemistry Education Research and Practice · doi:10.1039/c3rp20166j · Cited by 46
Facione, P. A. (1990) · Cited by 40
Stein, B., & Haynes, A. (2011) · Change The Magazine of Higher Learning · doi:10.1080/00091383.2011.550254 · Cited by 40
Towards a Definition of Representational Competence
Daniel, K. L., Bucklin, C. J., Leone, E. A., & Idema, J. L. (2018) · Models and modeling in science education · doi:10.1007/978-3-319-89945-9_1 · Cited by 40
Thinking about critical thinking: An industry perspective
Pearl, A. O., Rayner, G. M., Larson, I. C., & Orlando, L. (2018) · Industry and Higher Education · doi:10.1177/0950422218796099 · Cited by 28
Cole, R. S., Reynders, G. J., Ruder, S. M., Stanford, C., & Lantz, J. M. (2019) · Springer eBooks · doi:10.1007/978-981-13-6998-8_13 · Cited by 19
Self-citationFacilitating Strategies for Solving Work-Energy Problems in Graphical and Equational Representations
Nguyen, D.-H., Gire, E., Rebello, N. S., Singh, C., Sabella, M. S., & Rebello, N. S. (2010) · AIP conference proceedings · doi:10.1063/1.3515211 · Cited by 8
Hanson, D. M. (2008) · ACS symposium series · doi:10.1021/bk-2008-0994.ch002 · Cited by 7
Research works printed but not linked: 10
- American Chemical Society Committee on Professional Training. (2015). Unergraduate Professional Education in Chemistry: ACS Guidelines and Evaluation Procedures for Bachelor's Degree Programs. Retrieved from https://www.acs.org/content/dam/acsorg/about/governance/committees/training/2015-acs-guidelines-for-bachelors-degree-programs.pdf
- Biggs, J. (2003). Aligning teaching and assessing to course objectives. Teaching and learning in higher education: New trends and innovations, 2, 13–17.
- Brewer, C. A., & Smith, D. (2011). Vision and change in undergraduate biology education: a call to action. American Association for the Advancement of Science. DC: Washington.
- Gosser, D. K., Cracolice, M. S., Kampmeier, J. A., Roth, V., Strozak, V. S., & Varma-Nelson, P. (2001). Peer-led team learning: A guidebook: Prentice Hall Upper Saddle River, NJ.
- Gray, K., & Koncz, A. (2018). The key attributes employers seek on students' resumes. Retrieved from http://www.naceweb.org/about-us/press/2017/the-key-attributes-employers-seek-on-students-resumes/.
- Huitt, W. (1998). Critical thinking: an overview. In Educational psychology interactive Retrieved from http://www.edpsycinteractive.org/topics/cogsys/critthnk.html.
- Ramsden, P. (1997). The context of learning in academic departments. The experience of learning, 2, 198–216.
- Simonson, S. R. (Ed.). (2019). POGIL: An Introduction to Process Oriented Guided Inquiry Learning for Those Who Wish to Empower Learners. Sterling, VA: Stylus Publishing, LLC.
- The Royal Society. (2014). Vision for science and mathematics education: The Royal Society Science Policy Centre. London: England.
- Watson, G., & Glaser, E. M. (1964). Watson-Glaser Critical Thinking Appraisal Manual. New York, NY: Harcourt, Brace, and World.
Cited by
Sorted by citations
Purwaningsih, E., Sari, S. P., Sari, A. M., & Suryadi, A. (2020) · Jurnal Pendidikan IPA Indonesia · doi:10.15294/jpii.v9i4.26432 · Cited by 124
Hebebci, M. T., & Usta, E. (2022) · Participatory Educational Research · doi:10.17275/per.22.143.9.6 · Cited by 81
10 Guiding principles for learning in the laboratory
Seery, M. K., Agustian, H. Y., Christiansen, F. V., Gammelgaard, B., & Malm, R. H. (2023) · Chemistry Education Research and Practice · doi:10.1039/d3rp00245d · Cited by 101
Which professional skills do students learn in engineering team-based projects?
Picard, C., Hardebolle, C., Tormey, R., & Schiffmann, J. (2021) · European Journal of Engineering Education · doi:10.1080/03043797.2021.1920890 · Cited by 84
Chusni, M. M., Saputro, S., Rahardjo, S. B., & Suranto, S. (2021) · European Journal of Educational Research · doi:10.12973/eu-jer.10.3.1123 · Cited by 46
Metadata
- Authors
- Reynders, G., Lantz, J., Ruder, S. M., Stanford, C. L., & Cole, R. S.
- Affiliations
- University of Iowa (US)
- Drew University (US)
- Virginia Commonwealth University (US)
- Ball State University (US)
- Corresponding author
- Cole, Renée S.
- Venue
- International Journal of STEM Education
- Published
- 2020-03-09
- Volume
- 7
- Issue
- 1
- ISSN
- 2196-7822 (ISSN-L)
- OpenAlex
- W3031490426
- Cited by
- 174
- References
- 51
- Citations by year
- 2026: 21 · 2025: 29 · 2024: 31 · 2023: 41 · 2022: 26 · 2021: 13 · 2020: 13
- FWCI
- 35.23
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- International Journal of STEM Education (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Science Foundation (1524399, 1524936, 1524965)
- Topics
- Social Sciences › Education › Student Assessment and Feedback
- Social Sciences › Education › Education and Critical Thinking Development
- Social Sciences › Education › Educational Assessment and Pedagogy
- Keywords
- rubrics, critical thinking, information processing, process skills, undergraduate STEM education, formative feedback, constructive alignment, learning outcomes, written assessment, teaching assistants, active learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
