Project-Based STEM Learning Using Educational Robotics as the Development of Student Problem-Solving Competence
Open accessThe DOI matches its Crossref record; 21 of the 23 references it cites were found in a registry or an index; 3 of the 15 sources its claims cite were checked for support.
Abstract
The study focuses on teaching students using educational robots in the field of STEM. The study focused on the influence of project-based teaching on the development of student competences, especially problem-solving competences. The research part of the study describes the conducted pedagogical experiment—teaching pupils the programming of educational robots. The experiment compared two groups of students in the 8th grade of elementary school, using the “Skills for Life” test, which is used to test student competencies. Project-based teaching in STEM fields using educational robotics is very popular among students and, according to research results, has an impact on the development of student competencies. The results of the presented study clearly demonstrate the positive influence of project-based teaching using educational robots on the development of student competencies, especially the important key competencies for solving problems. The key competence to solve problems is applicable both in the areas of STEM education, but also in the everyday life of the student.
Overview
CoChat Summary
TL;DR
In an 8th-grade pedagogical experiment, project-based programming with Lego Mindstorms EV3 improved problem-solving competence compared with traditional programming instruction. Both hypothesis tests were statistically significant (Mann-Whitney U = 30 and U = 38 vs. critical value 55; Fisher exact p = 0.046).
Key findings
- Students taught by project-based learning of robotic-kit programming were more successful on the problem-solving post-test than students taught the traditional way: Mann-Whitney U = 30, critical w₀.₀₅;₁₄;₁₄ = 55, so 30 < 55; Fisher exact test p = 0.046, rejecting H0₁.
- The experimental group showed a statistically significant pre-test to post-test improvement in problem-solving competence: Mann-Whitney U = 38, critical w₀.₀₅;₁₄;₁₄ = 55, so 38 < 55; Fisher exact test p = 0.046, rejecting H0₂.
- The study concludes that project-based teaching using educational robots had a positive influence on the development of student competencies, especially problem-solving competence, which the author describes as applicable in STEM education and everyday life.
Methods
- Design: pedagogical experiment with pre-test and post-test, comparing an experimental group and a control group of 8th-grade elementary school pupils; the text reports n = 14 and m = 14 (total 28).
- Setting and intervention: one urban primary school selected by availability and COVID-19-related possibilities; both groups used Lego Mindstorms EV3 kits and LEGO MINDSTORMS Education EV3, with the same topics and continuity; the experimental group used project-based learning in four phases (planning, implementation, presentation, evaluation), while the control group received teacher-managed traditional programming lessons.
- Measures and analysis: the adaptive online “Skills for Life” test from Scio, covering working with information, relationships and communication, and problem solving, with 3 × 15 questions and a recommended 60 min (most pupils finished within 45 min); after normality testing (Shapiro-Wilk and Kolmogorov-Smirnov) showed non-normal data, the author used the nonparametric Mann-Whitney test at α = 0.05, validated with Fisher’s combinatorial test.
Limitations
- Stated: the research sample was selected according to availability and current possibilities in elementary schools under COVID-19 government regulations; the study was implemented in one urban primary school, and the author notes that implementation was demanding and will be the basis for further research.
- Stated: project teaching was new for some pupils and sometimes more demanding, especially from a time perspective; the author also states there is no other suitable validated test in the pupils’ mother tongue and proposes a longitudinal study.
- Evident from the text: the sample is very small (14 pupils per group, total 28), limited to one school and one grade, with a short teaching period in May and June of the 2020/2021 school year; no effect sizes or confidence intervals are reported, and some hypothesis/statistical labels appear inconsistent (e.g., the H2 section repeats H0₁/H A₁ labels).
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
2 Valid · 2 Not valid · 6 Not checked · 22 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“STEM is effective in student learning [6] and is not only about knowledge of specific topics, but develops students' key skill competencies [6][7][8][9][10].”
§ Project-Based STEM Learning Using Educational Robotics as the Development of Student Problem-Solving Competence > 1.1. STEM and Educational Robotics
What the sources say
- Partly supportedcites [6] Margot, 2019, Teachers’ Perception of STEM Integration and Education: A Systematic Literature Review, Int. J. STEM Educ. sourceExperimental — accuracy not yet measured
The “STEM education and talent development” section (p. 2) lists twenty-first-century competencies among the goals of STEM education. “Cross-curricular integration” (p. 9) reports teachers’ views that STEM gives students problem-solving skills. These passages support the focus on skills beyond subject knowledge, but they describe goals and perceptions rather than establish that STEM education develops those competencies. (LLM, conf=0.84)
- Supportedcites [7] Wahono, 2020, Evidence of STEM Enactment Effectiveness in Asian Student Learning Outcomes, Int. J. STEM Educ. source
The Introduction (p. 1) identifies problem-solving, higher-order thinking and collaboration as focuses of STEM learning, beyond subject knowledge. The Conclusion (p. 15) reports that STEM education improved higher-order thinking skills among students in Asia, supporting the attributed claim about developing key competencies. (LLM, conf=0.90)
- Not checked — no full text of the sourcecites [8] Sorby, 2018, Does Spatial Skills Instruction Improve STEM Outcomes?, The Answer Is ‘Yes’. Learn. Individ. Differ. source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For our table, the value of the resulting probability is p = 0.046, which is lower than α = 0.05; therefore, we rejected hypothesis H 02 and accepted the alternative hypothesis H A2 , and it is true that there is a statistically significant difference between the means obtained in the two groups.”
- Also stated in: § Statistical verification of the research hypothesis H1 · plain
“The result of this verification is that there is a statistically significant difference between the pre-test and post-test for the students in the experimental group.”
§ Project-Based STEM Learning Using Educational Robotics as the Development of Student Problem-Solving Competence > H A2 :
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Statistical verification of the research hypothesis H1 · plain
None of the sources cited for this claim have been checked yet.
In the paper
“when verifying the normality of the data using the Shapiro-Wilk normality test [43,44] and the Kolmogorov-Smirnov test [45,46], we found that these are not normally distributed data and the Student's t-test cannot be used for these data.”
§ Project-Based STEM Learning Using Educational Robotics as the Development of Student Problem-Solving Competence > 2.6. Statistical Processing of the Obtained Data
What the sources say
- Not checked — no full text of the sourcecites [43] Shapiro, 1965, An Analysis of Variance Test for Normality (Complete Samples), Biometrika source
- Not matchedcites [44] Statistics Kingdom (2021, December 20). Shapiro-Wilk Test Calculator, Available online: https://www.statskingdom.com/320ShapiroWilk.html.
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites [45] Stangroom, J. (2021, December 20). The Kolmogorov-Smirnov Test of Normality, Available online: https://www.socscistatistics.com/tests/kolmogorov/default.aspx.
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites [46] Chráska, M. (2007). Metody Pedagogického Výzkumu: Základy Kvantitativního Výzkumu, Grada. Pedagogika.
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Through the Skills for Life test that was used to test problem-solving competence, the experimental group showed better results.”
§ Project-Based STEM Learning Using Educational Robotics as the Development of Student Problem-Solving Competence > Statistical verification of the research hypothesis H1
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The graph in Figure 3 shows the distribution of levels obtained by the students in the problem-solving area of the test.The graph shows the differences between the control and experimental groups of pupils.”
§ Project-Based STEM Learning Using Educational Robotics as the Development of Student Problem-Solving Competence > 3.1. Pre-Test and Post-Test
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
8 more claims
Create a free account to see all 32 claims and to verify the 2 references still open.
- Probably not a substantial claim. 15
- It may state two separate claims as one. 6
- The quote only partly states this claim. 2
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Project-Based STEM Learning Using Educational Robotics as the Development of Student Problem-Solving Competence > 30 < 55
“we had to proceed using Fisher's combinatorial test for the four-field table [46].”
§ Project-Based STEM Learning Using Educational Robotics as the Development of Student Problem-Solving Competence > References
“46. Statistics Kingdom. Shapiro-Wilk Test Calculator.”
Citations
Cites
An analysis of variance test for normality (complete samples)
SHAPIRO, S. S., & WILK, M. B. (1965) · Biometrika · doi:10.1093/biomet/52.3-4.591 · Cited by 23,480
An Analysis of Variance Test for Normality (Complete Samples)
Shapiro, S. S., & Wilk, M. B. (1965) · Biometrika · doi:10.2307/2333709 · Cited by 23,469
Teachers’ Perception of STEM Integration and Education: A Systematic Literature Review
Margot, K. C., & Kettler, T. (2019) · International Journal of STEM Education · doi:10.1186/s40594-018-0151-2 · Cited by 992
What are we talking about when we talk about STEM education? A review of literature
Martín‐Páez, T., Aguilera, D., Perales‐Palacios, F. J., & Vílchez‐González, J. M. (2019) · Science Education · doi:10.1002/sce.21522 · Cited by 566
Bybee, R. W. (2010) · Science · doi:10.1126/science.1194998 · Cited by 532
Research and trends in STEM education: a systematic review of journal publications
Li, Y., Wang, K., Xiao, Y., & Froyd, J. E. (2020) · International Journal of STEM Education · doi:10.1186/s40594-020-00207-6 · Cited by 494
Kilpatrick, W. H. (1918) · Teachers College Record: The Voice of Scholarship in Education · doi:10.1177/016146811801900404 · Cited by 431
Evidence of STEM Enactment Effectiveness in Asian Student Learning Outcomes
Wahono, B., Lin, P.-L., & Chang, C.-Y. (2020) · International Journal of STEM Education · doi:10.1186/s40594-020-00236-1 · Cited by 286
(2016) · Griffith Research Online · Cited by 215
Does spatial skills instruction improve STEM outcomes? The answer is ‘yes’
Sorby, S., Veurink, N., & Streiner, S. (2018) · Learning and Individual Differences · doi:10.1016/j.lindif.2018.09.001 · Cited by 179
Research works printed but not linked: 1
- Weinberg, S., and Abramowitz, S. (2008). Statistics Using SPSS: An Integrative Approach, Cambridge University Press.
Cited by
Assessing teacher readiness for educational robotics integration in primary and preschool education
Gavrilas, L., Kotsis, K. Τ., & Papanikolaou, M.-S. (2024) · Education 3-13 · doi:10.1080/03004279.2023.2300699 · Cited by 29
“This methodology hinges on involving students in real-time problem-solving, fostering their critical thinking and engagement continuously and collaboratively (Abdurrahman, Achmad, and Nurulsari 2019; Coufal 2022; Ford, Dack, and Prejean 2006).”
Suhirman, S., & Prayogi, S. (2023) · Jurnal Penelitian Pendidikan IPA · doi:10.29303/jppipa.v9i8.4715 · Cited by 41
“The application of PjBL in STEM can improve problem-solving competencies (Coufal, 2022), student motivation, and interest in learning (Domenici, 2022).”
BackgroundGavrilas, L., & Kotsis, K. Τ. (2024) · Contemporary Mathematics and Science Education · doi:10.30935/conmaths/14384 · Cited by 21
“The project method (Bertacchini et al., 2022), problem-based learning (PBL) (Coufal, 2022; Hmelo-Silver, 2004), and inquiry-based learning (IBL) methods (Abdurrahman et al., 2019; Yuliati et al., 2018), are identified as ideal modalities for advancing STEM education (Erdogan & Stuessy, 2016).”
MethodTorres, I., & Inga, E. (2025) · Information · doi:10.3390/info16020096 · Cited by 33
Gavrilas, L., Kotsis, K. Τ., & Papanikolaou, M.-S. (2023) · Pedagogical Research · doi:10.29333/pr/14049 · Cited by 15
“This approach relies on engaging students in real-time problem-solving, nurturing their continuous and collaborative critical thinking and involvement (Bertacchini et al., 2022; Coufal, 2022; Sahin, 2013).”
Metadata
- Authors
- Coufal, P.
- Affiliations
- University of Hradec Králové (CZ)
- Corresponding author
- Coufal, Petr
- ORCID
- Coufal, Petr
- Venue
- Mathematics
- Publisher
- MDPI AG
- Published
- 2022-12-06
- Volume
- 10
- Issue
- 23
- Pages
- 4618
- ISSN
- 2227-7390 (ISSN-L)
- OpenAlex
- W4311723994
- Cited by
- 55
- References
- 23
- Citations by year
- 2026: 15 · 2025: 15 · 2024: 15 · 2023: 10
- FWCI
- 3.29
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Mathematics (publishedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Specific research project of University of Hradec Kralove, Faculty of Education (2126)
- Topics
- Social Sciences › Education › Educational Robotics and Engineering
- Social Sciences › Education › Problem and Project Based Learning
- Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
- Keywords
- educational robotics, project-based learning, STEM education, problem-solving skills, programming education, core competencies, eighth grade students, elementary school
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.