CoChat
Sign Up
Cited by 28Problem and Project Based LearningSTEM EducationScience Education and Pedagogy

A Meta-Analysis to Gauge the Effectiveness of STEM Informal Project-Based Learning: Investigating the Potential Moderator Variables

Open access
Santhosh, Malavika, Farooqi, Hafsa, Ammar, Mohammad, Siby, Nitha, Bhadra, Jolly, Al-Thani, Noora J., Sellami, Abdellatif, Fatima, Noshin +1 more·2023·Journal of Science Education and Technology
DOI 10.1007/s10956-023-10063-y
Open this paper in CoChat — free accountOpen PDF

Abstract

In recent decades, informal project-based learning (PjBL) for science, technology, engineering, and mathematics (STEM) education has gained immense significance. Though some meta-analytical studies have reported the effectiveness of PjBL models, none of them has been explicit for informal PjBL models executed for school students in STEM education. Therefore, this meta-analytical study sought to investigate the effect of the informal PjBL model on students’ learning gains when compared to the traditional classroom setting. The study also intends to explore the effect of potential moderator variables that might impact the effectiveness of the model. For this, a systematic review aligning to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) methodology has been employed, by incorporating 26 peer-reviewed empirical studies, encompassing data from 3202 students from eight countries. The meta-analytical results revealed overall moderate effectiveness (d = 0.248, p < 0.001), indicating that informal PjBL has moderately significant effects on students’ gains when compared to the traditional classroom setting. Correspondingly the moderator analysis revealed that the overall effectiveness of informal PjBL was influenced by the teaching model, assessment method, students’ group size, and course duration. However, the participants’ educational level and study location (country) did not emerge as being significant. This study contributes to the bulk of literature that can help academicians to design, develop and implement effective STEM-based informal PjBL models, with special consideration for influential variables (teaching method, assessment method, grade, location, course duration, group size, and subject area).

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/s10956-023-10063-y

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 57

  1. Al-Kadri, H. M., Al-Moamary, M. S., Roberts, C., & Van der Vleuten, C. P. (2012). Exploring assessment factors contributing to students' study strategies: literature review. Medical teacher, 34(sup1), S42- S50. https:// doi. org/ 10. 3109/ 01421 59X. 2012. 656756 · DOI
  2. Apedoe, X. S., Ellefson, M. R., & Schunn, C. D. (2012). Learning together while designing: Does group size make a difference? Journal of Science Education and Technology, 21(1), 83-94. https:// doi. org/ 10. 1007/ s10956-011-9284-5 · DOI
  3. Awad, N. (2021). Exploring STEM integration: assessing the effective- ness of an interdisciplinary informal program in fostering students' performance and inspiration. Research in Science & Technological Education, 1-25. https:// doi. org/ 10. 1080/ 02635 143. 2021. 19318 · DOI
  4. Ayaz, M. F., & Söylemez, M. (2015). The effect of the project-based learning approach on the academic achievements of the students in science classes in Turkey: A meta-analysis study. Egitim ve Bilim, 40(178). https:// doi. org/ 10. 15390/ EB. 2015. 4000 · DOI · Paper page
  5. Baran, E., Canbazoglu Bilici, S., Mesutoglu, C., & Ocak, C. (2019). The impact of an out-of-school STEM education program on stu- dents' attitudes toward STEM and STEM careers. School Science and Mathematics, 119(4), 223-235. https:// doi. org/ 10. 1111/ ssm. 12330 · DOI
  6. Balemen, N., & Keskin, M. Ö. (2018). The effectiveness of Project- Based Learning on science education: A meta-analysis search. International Online Journal of Education and Teaching, 5(4), 849-865. Retrieved April 15, 2022, from https:// eric. ed. gov/? id= EJ125 0564
  7. Bertucci, A., Conte, S., Johnson, D. W., & Johnson, R. T. (2010). The impact of size of cooperative group on achievement, social support, and self-esteem. The Journal of General Psychology: Experimental, Psychological, and Comparative Psychology, 137(3), 256-272. https:// doi. org/ 10. 1080/ 00221 309. 2010. 484448 · DOI
  8. Bicer, A., & Lee, Y. (2019). Effect of STEM PBL embedded informal learning on student interest in STEM majors and careers. Jour- nal of Mathematics Education, 12(1), 57-73. https:// doi. org/ 10. 26711/ 00757 71527 90038 · DOI
  9. Bielefeldt, A. R. (2013). Pedagogies to achieve sustainability learning outcomes in civil and environmental engineering students. Sus- tainability, 5(10), 4479-4501. https:// doi. org/ 10. 3390/ su510 4479 · DOI
  10. Bokor, J. R., Landis, J. B., & Crippen, K. J. (2014). High school stu- dents' learning and perceptions of phylogenetics of flowering plants. CBE-Life Sciences Education, 13(4), 653-665. https:// doi. org/ 10. 1187/ cbe. 14-04-0074 · DOI
  11. Borenstein, M., Hedges, L. v., Higgins, J. P. T., & Rothstein, H. R. (2009). Introduction to meta-analysis. Introduction to Meta-Analysis, 1-421. https:// doi. org/ 10. 1002/ 97804 70743 386 · DOI · Paper page
  12. Calabrese, J. E., & Capraro, R. M. (2021). The autonomy of informal STEM and benefits of andragogy with gifted children. Journal of Research in Innovative Teaching & Learning. https:// doi. org/ 10. 1108/ JRIT-08-2021-0060 · DOI
  13. Chen, C. H., & Yang, Y. C. (2019). Revisiting the effects of project-based learning on students' academic achievement: A meta-analysis investigating moderators. Educational Research Review, 26, 71-81. https:// doi. org/ 10. 1016/j. edurev. 2018. 11. 001 · DOI · Paper page
  14. Chounta, I. A., Manske, S., & Hoppe, H. U. (2017). "From Making to Learning": Introducing Dev Camps as an educational paradigm for Re-inventing Problem-based Learning. International Journal of Educational Technology in Higher Education, 14(1), 1-15. https:// doi. org/ 10. 1186/ s41239-017-0061-2 · DOI
  15. Cilliers, F. J., Schuwirth, L. W., Herman, N., Adendorff, H. J., & van der Vleuten, C. P. (2012). A model of the pre-assessment learning effects of summative assessment in medical education. Advances in Health Sciences Education, 17, 39-53. https:// doi. org/ 10. 1007/ s10459-011-9292-5 · DOI
  16. Cochran, W. G. (1954). Some methods for strengthening the common χ 2 tests. Biometrics, 10(4), 417-451. https:// doi. org/ 10. 2307/ 30016 16 · DOI · Paper page
  17. Cohen, J. (2013). Statistical power analysis for the behavioral sci- ences. Routledge. https:// doi. org/ 10. 4324/ 97802 03771 587 · DOI · Paper page
  18. Collins, M. A., Totino, J., Hartry, A., Romero, V. F., Pedroso, R., & Nava, R. (2020). Service-learning as a lever to support STEM engagement for underrepresented youth. Journal of Experiential Education, 43(1), 55-70. https:// doi. org/ 10. 1177/ 10538 25919 887407 · DOI
  19. Covert, H., Ilunga Tshiswaka, D., Ramkissoon, I., Sisskin, E., Lichtveld, M., & Wickliffe, J. (2019). Assessing science motivation among high school students participating in a supplemental science pro- gramme: The emerging scholars environmental health sciences academy. International Journal of Science Education, 41(17), 2508-2523. https:// doi. org/ 10. 1080/ 09500 693. 2019. 16893 08 · DOI
  20. DerSimonian, R., & Laird, N. (2015). Meta-analysis in clinical trials revisited. Contemporary Clinical Trials, 45, 139-145. https:// doi. org/ 10. 1016/j. cct. 2015. 09. 002 · DOI · Paper page
  21. Duval, S., & Tweedie, R. (2000). Trim and fill: A simple funnel- plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics, 56(2), 455-463. https:// doi. org/ 10. 1111/J. 0006-341X. 2000. 00455. · DOI · Paper page
  22. X · DOI · Paper page
  23. Gao, X., Li, P., Shen, J., & Sun, H. (2020). Reviewing assessment of student learning in interdisciplinary STEM education. Interna- tional Journal of STEM Education, 7(1), 1-14. https:// doi. org/ 10. 1186/ s40594-020-00225-4 · DOI · Paper page
  24. Gerretson, H., & Golson, E. (2005). Synopsis of the use of course- embedded assessment in a medium sized public university's general education program. The Journal of General Educa- tion, 54(2), 139-149. https:// doi. org/ 10. 2307/ 27798 013 · DOI
  25. Ghrayeb, O., Damodaran, P., & Vohra, P. (2011). Art of triangula- tion: An effective assessment validation strategy. Global Journal of Engineering Education, 13(3), 96-101. Retrieved April 15, 2022, from http:// www. wiete. com. au/ journ als/ GJEE/ Publi sh/ vol13 no3/ 07-Vohra-P. pdf
  26. Habig, B., & Gupta, P. (2021). Authentic STEM research, practices of science, and interest development in an informal science edu- cation program. International Journal of STEM Education, 8(1), 1-18. https:// doi. org/ 10. 1186/ s40594-021-00314 · DOI
  27. Hirsch, L. S., Berliner-Heyman, S., & Cusack, J. L. (2017). Introduc- ing middle school students to engineering principles and the engineering design process through an academic summer pro- gram. International Journal of Engineering Education, 33(1), 398-407. Retrieved April 15, 2022, from https:// www. njit. edu/ preco llege/ sites/ njit. edu. preco llege/ files/ Inter natio nal_ Journ al_ of_ Engin eering_ Educa tion_ Speci al_ Editi on_ Hirsch. pdf
  28. Innes, T., Johnson, A. M., Bishop, K. L., Harvey, J., & Reisslein, M. (2012). The Arizona Science Lab (ASL): Fieldtrip based STEM outreach with a full engineering design, build, and test cycle. Global Journal of Engineering Education, 14(3), 225-232. Retrieved April 15, 2022, from https:// cites eerx. ist. psu. edu/ viewd oc/ downl oad? doi= 10.1. 1. 667. 7339& rep= rep1& type= pdf
  29. Johnson, D. R., Renzulli, L., Bunch, J., & Paino, M. (2013). Eve- ryday observations: Developing a sociological perspective through a portfolio term project. Teaching Sociology, 41(3), 314-321. https:// doi. org/ 10. 1177/ 00920 55X13 480642 · DOI
  30. Kim, Y. J., Murai, Y., & Chang, S. (2021). Implementation of embed- ded assessment in maker classrooms: Challenges and opportu- nities. Information and Learning Sciences, 122(3/4), 292-314. https:// doi. org/ 10. 1108/ ILS-08-2020-0188 · DOI
  31. Kokotsaki, D., Menzies, V., & Wiggins, A. (2016). Project-based learning: A review of the literature. Improving Schools, 19(3), 267-277. https:// doi. org/ 10. 1177/ 13654 80216 659733 · DOI · Paper page
  32. Kwon, H., Capraro, R. M., & Capraro, M. M. (2021). When I believe, I can: Success STEMs from my perceptions. Canadian Journal of Science, Mathematics and Technology Education, 21(1), 67-85. https:// doi. org/ 10. 1007/ s42330-020-00132-4 · DOI
  33. Kooloos, J. G. M., Klaassen, T., Vereijken, M., van Kuppeveld, S., Bolhuis, S., & Vorstenbosch, M. (2011). Collaborative group work: Effects of group size and assignment structure on learning gain, student satisfaction and perceived participation. Medical Teacher, 33(12), 983-988. https:// doi. org/ 10. 3109/ 01421 59X. 2011. 588733 · DOI
  34. Larmer, J., & Mergendoller, J. R. (2010). Seven essentials for project- based learning. Educational leadership, 68(1), 34-37. Retrieved April 15, 2022, from https:// d1wqt xts1x zle7. cloud front. net/ 64271 329/ Seven_ Essen tials_ for_ Proje ct_ Based_ Learn-with-cover-page- v2. pdf? Expir es= 16635 05928 & Signa ture= aaHCd Bmj9V LXYn5 SKVMq VvwoN 1DI6-0h4BM uNqAb L3RzI qlVBJ qkIpB r4u3W mnByr Cmcyb TrqN1 4Ziuj dJy2L zkQdr I8auJ 6UkFK unwtHU- 36XYc 9a3~kHpUp xB43W 6l3Bz WrWmD 4jbPx EHpBx QZEn8 QTBp3 gyBBX UPe7d 8XWqu meOsO IRCGG qMmXB x~s-WTMW- LW1~hP-XQvc9-lK1hg CCi69 9O1rA tOIlR 9pvvy2-9fN0W nAp- umsCc VqE64 XoDDF YJbDh Bd6E5 KDSvY DrVhv rjDko 3QVyZ XP6oD 9YPP8 cNi-9oabe 6CTq4 I58fx 7Txfn oAnOT 2kz2O 8TqnL 65vA__ & Key-Pair-Id= APKAJ LOHF5 GGSLR BV4ZA
  35. Lu, S. Y., Lo, C. C., & Syu, J. Y. (2021). Project-based learning ori- ented STEAM: The case of micro-bit paper-cutting lamp. Interna- tional Journal of Technology and Design Education, 1-23. https:// doi. org/ 10. 1007/ s10798-021-09714-1 · DOI
  36. Marriott, P., & Lau, A. (2008). The use of on-line summative assess- ment in an undergraduate financial accounting course. Journal of Accounting Education, 26(2), 73-90. https:// doi. org/ 10. 1016/j. jacce du. 2008. 02. 001 · DOI
  37. Martell, K., & Calderon, T. (2005). Assessment in business schools: What it is, here we are, and where we need to go now. In K. Martell & T. Calderon (Eds.), Assessment of student learning in business schools: best practices each step of the way (pp. 1-22). Tallahassee FL: Association for Institutional Research.
  38. Mateos-Nunez, M., Martinez-Borreguero, G., & Naranjo-Correa, F. L. (2020). Learning science in primary education with STEM work- shops: Analysis of teaching effectiveness from a cognitive and emo- tional perspective. Sustainability, 12(8), 3095. https:// doi. org/ 10. 3390/ SU120 83095 · DOI
  39. Mohd Shahali, E. H., Halim, L., Rasul, M. S., Osman, K., & Mohamad Arsad, N. (2019). Students' interest towards STEM: A longitudi- nal study. Research in Science & Technological Education, 37(1), 71-89. https:// doi. org/ 10. 1080/ 02635 143. 2018. 14897 · DOI
  40. Moreno, N. P., Tharp, B. Z., Vogt, G., Newell, A. D., & Burnett, C. A. (2016). Preparing students for middle school through after-school STEM activities. Journal of Science Education and Technology, 25(6), 889-897. https:// doi. org/ 10. 1007/ s10956-016-9643-3 · DOI
  41. Nadelson, L. S., Jemison, R. C., Soto, E., & Warner, D. L. (2021). Cul- tivating a New "SEED": From an on-Ground to Online Chemistry Summer Camp. Journal of Chemical Education, 99(1), 129-139. https:// doi. org/ 10. 1021/ acs. jchem ed. 1c002 80 · DOI
  42. Newton, K. J., Leonard, J., Buss, A., Wright, C. G., & Barnes-John- son, J. (2020). Informal STEM: Learning with robotics and game design in an urban context. Journal of Research on Technology in Education, 52(2), 129-147. https:// doi. org/ 10. 1080/ 15391 523. 2020. 17132 · DOI
  43. Nugent, G., Barker, B., Grandgenett, N., & Adamchuk, V. I. (2010). Impact of robotics and geospatial technology interventions on youth STEM learning and attitudes. Journal of Research on Tech- nology in Education, 42(4), 391-408. https:// doi. org/ 10. 1080/ 15391 523. 2010. 10782 557 · DOI · Paper page
  44. Ramdhayani, H. G., Purwoko, A. A., & Muntari, M. (2019, Octo- ber). Meta analysis: the effect of applying project-based learning models to students' science process skills. In Journal of Physics: Conference Series, 1321(3), 032089. IOP Publishing. https:// doi. org/ 10. 1088/ 1742-6596/ 1321/3/ 032089 · DOI
  45. Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86(3), 638-641. https:// doi. org/ 10. 1037/ 0033-2909. 86.3. 638 · DOI · Paper page
  46. Schellens, T., van Keer, H., de Wever, B., & Valcke, M. (2007). Script- ing by assigning roles: Does it improve knowledge construction in asynchronous discussion groups? International Journal of Com- puter-Supported Collaborative Learning, 2(2-3). https:// doi. org/ 10. 1007/ s11412-007-9016-2 · DOI
  47. Shpeizer, R. (2019). Towards a successful integration of project- based learning in higher education: Challenges, technologies and methods of implementation. Universal Journal of Educational Research, 7(8), 1765-1771. https:// doi. org/ 10. 13189/ ujer. 2019. 070815 · DOI
  48. Sivia, A., MacMath, S., Novakowski, C., & Britton, V. (2019). Examin- ing student engagement during a project-based unit in secondary science. Canadian Journal of Science, Mathematics and Technol- ogy Education, 19(3), 254-269. https:// doi. org/ 10. 1007/ S42330- 019-00053-X/ TABLES/8 · DOI
  49. Smit, R., Robin, N., De Toffol, C., & Atanasova, S. (2021). Industry- school projects as an aim to foster secondary school students' interest in technology and engineering careers. International Jour- nal of Technology and Design Education, 31(1), 61-79. https:// doi. org/ 10. 1007/ s10798-019-09538-0 · DOI
  50. Stevens, S., Andrade, R., & Page, M. (2016). Motivating young Native American students to pursue STEM learning through a culturally relevant science program. Journal of Science Educa- tion and Technology, 25(6), 947-960. https:// doi. org/ 10. 1007/ s10956-016-9629-1 · DOI
  51. Tekbıyık, A., Bulut, D. B., & Sandalcı, Y. (2022). Effects of a summer robotics camp on students' STEM career interest and knowledge structure. Journal of Pedagogical Research, 6(2), 91-109. https:// doi. org/ 10. 33902/ JPR. 20221 2606 · DOI
  52. Todd, B. L., & Zvoch, K. (2019). The effect of an informal science intervention on middle school girls' science affinities. Interna- tional Journal of Science Education, 41(1), 102-122. https:// doi. org/ 10. 1080/ 09500 693. 2018. 15340 · DOI
  53. Todd, B., & Zvoch, K. (2017). Exploring girls' science affinities through an informal science education program. Research in Science Education, 49(6), 1647-1676. https:// doi. org/ 10. 1007/ s11165-017-9670-y · DOI
  54. Tuunila, R., & Pulkkinen, M. (2015). Effect of continuous assessment on learning outcomes on two chemical engineering courses: Case study. European Journal of Engineering Education, 40(6), 671- 682. https:// doi. org/ 10. 1080/ 03043 797. 2014. 10018 · DOI
  55. Vela, K. N., Pedersen, R. M., & Baucum, M. N. (2020). Improving perceptions of STEM careers through informal learning environ- ments. Journal of Research in Innovative Teaching & Learning, 13(1), 103-113. https:// doi. org/ 10. 1108/ jrit-12-2019-0078 · DOI
  56. Yin, Y., Hadad, R., Tang, X., & Lin, Q. (2020). Improving and assess- ing computational thinking in maker activities: The integration with physics and engineering learning. Journal of Science Edu- cation and Technology, 29(2), 189-214. https:// doi. org/ 10. 1007/ s10956-019-09794-8 · DOI
  57. Zhou, N., Pereira, N. L., George, T. T., Alperovich, J., Booth, J., Chandrasegaran, S., & Ramani, K. (2017). The influence of toy design activities on middle school students' understanding of the engineering design processes. Journal of Science Educa- tion and Technology, 26(5), 481-493. https:// doi. org/ 10. 1007/ s10956-017-9693-1 · DOI

Cited by

Sorted by citations

Authors
Santhosh, M., Farooqi, H., Ammar, M., Siby, N., Bhadra, J., Al-Thani, N. J., Sellami, A., Fatima, N., & Ahmad, Z.
Affiliations
  • Qatar University (QA)
  • UCSI University (MY)
Corresponding author
Ahmad, Zubair
ORCID
Venue
Journal of Science Education and Technology
Publisher
Springer Science and Business Media LLC
Published
2023-07-17
Volume
32
Issue
5
Pages
671-685
ISSN
  • 1059-0145 (ISSN-L)
  • 1573-1839
DOI
10.1007/s10956-023-10063-y
OpenAlex
W4384524918
Cited by
28
References
2
Citations by year
2026: 13 · 2025: 11 · 2024: 2 · 2023: 2
FWCI
1.72
Type
journal article
Language
en
Open access
Licence
Locations
  • Journal of Science Education and Technology (publishedVersion, cc-by)
  • Figshare (submittedVersion, cc-by)
  • Qatar University QSpace (Qatar University) (submittedVersion)
  • Greater South Information System
Funders
Qatar University
Topics
  • Social Sciences › Education › Problem and Project Based Learning
  • Social Sciences › Education › STEM Education
  • Social Sciences › Education › Science Education and Pedagogy
Keywords
STEM education, project-based learning, learning gains, traditional classroom comparison, moderators, teaching methods, assessment methods, group size, course duration, school students

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review