CoChat
Sign Up
Cited by 7Simulation Techniques and ApplicationsExperimental Learning in EngineeringEngineering Education and Pedagogy

An Assessment of Simulation-Based Learning Modules in an Undergraduate Engineering Economy Course

Open access
Negahban, Ashkan, Ashour, Omar, Nowparvar, Mahgol, Knight, Daniel, Ozden, Sabahattin, Delgoshaei, Parhum·2024·2022 ASEE Annual Conference & Exposition Proceedings
DOI 10.18260/1-2--41913
Open this paper in CoChat — free accountOpen PDF

Abstract

simulation methods, primarily agent-based and discrete-event simulation.

AI snapshot

Written from the paper’s full text.

TL;DR

This paper implements and assesses Immersive Simulation-Based Learning (ISBL) modules in an undergraduate engineering economy course, comparing an intervention group using ISBL assignments with a control group using traditional homework. It finds statistically significant gains in motivation (Confidence) and experiential learning (Utility), with no significant difference in self-assessed learning.

Key findings

  • Motivation (IMMS): the ISBL group had higher mean and median on all constructs; “Confidence” was highly statistically significant (p = 0.000), while “Relevance” barely missed significance (p = 0.051), and Attention (p = 0.126), Satisfaction (p = 0.158), and Overall (p = 0.335) were not significant.
  • Experiential learning: “Utility” was significantly higher for ISBL (p = 0.009), while “Environment” (p = 0.753) and Overall (p = 0.762) were not significant.
  • Engineering identity: a statistical difference appeared for “Recognition,” but the Control group performed better; authors attribute this to the ISBL group having more non-engineering majors (8.2% vs. 3.1%) and the short intervention. Self-assessment based on Bloom’s taxonomy showed no significant differences (overall p = 0.792), with both groups reporting the same median for all topics.
  • Qualitative interviews with ten students produced themes of “Real World Context,” “Engagement,” and “Learning about a Career,” supporting three of the four hypotheses.

Methods

  • Controlled experiment in an online undergraduate engineering economy course at Penn State Behrend, offered Fall 2020 and Spring 2021; control and intervention groups used the same instructor, material, and delivery, with the intervention replacing traditional homework with four ISBL modules (restaurant, manufacturing assembly plant, warehouse, airport terminal) built in Simio as 3D, VR-compatible discrete-event simulations.
  • Instruments included demographics, BFI-10 personality test, IMMS motivation survey, experiential learning survey, engineering identity survey, Bloom’s taxonomy self-assessment, and interviews with ten student volunteers; baseline tests found no significant BFI or GPA differences (GPA control mean 3.111 vs. ISBL 2.917, p = 0.132).
  • Quantitative comparisons used two-sample t-tests at a 5% significance level; qualitative interview notes were analyzed using Grounded Theory techniques.

Limitations

  • The authors state the intervention’s scope and duration were too limited/short to significantly affect engineering identity, requiring a longitudinal study over an extended period and multiple courses; they also state the self-assessment survey failed to capture a statistical difference and recommend alternative instruments to measure learning, plus additional experiments to assess ISBL’s impact on learning.
  • The authors note that “Relevance” had a p-value of 0.051, just over the cutoff, and deserves further investigation with additional data.
  • Evident from the text: the qualitative assessment relies on ten student volunteers, and total group sizes are not reported in the provided text, limiting generalizability; the engineering identity comparison is also confounded by the higher proportion of non-engineering majors in the ISBL group.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.18260/1-2--41913

Cites

23 of 23 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 19 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    A two-sample t-test at a 5% significance level indicates no significant statistical difference between the two groups in terms of average GPA.

    “As shown in Table 3, a twosample t-test at a 5% significance level indicates no significant statistical difference between the two groups in terms of the average GPA (i.e., we fail to reject H 0 : µ 𝐺𝑃𝐴 𝐶𝑜𝑛𝑡𝑟𝑜𝑙 -µ 𝐺𝑃𝐴 𝐼𝑆𝐵𝐿 = 0).”
    Quote in the paperSubstantial claim
  • Findingown data

    Qualitative themes emerged from the data that support three of the four hypotheses and findings from the quantitative analysis.

    “Themes emerged from the data which support three of the four hypotheses and findings from the results of the quantitative analysis.”
    Quote in the paperSubstantial claim
  • Findingown data

    The GPA distribution is similar for the two groups of students being compared.

    “Figure 4 shows that the GPA distribution is also similar for the two groups of students being compared.”
    Quote in the paperSubstantial claim
  • Findingown data

    For Relevance, the test barely fails to reject the null hypothesis with a p-value of 0.051, just over the cut-off point of 0.05.

    “For “Relevance”, we barely fail to reject the null hypothesis with a p-value of 0.051, just over the cut-off point of 0.05, deserving of further investigation with additional data.”
    Quote in the paperNot a substantial claim
  • Findingown data

    The self-assessment survey failed to capture a statistical difference between the control and intervention groups.

    “Our self-assessment survey failed to capture a statistical difference between the control and intervention groups, hence we would recommend use of alternative instruments to measure learning.”
    Quote in the paperSubstantial claim
  • May combine two claims. 18
  • Classified as probably not a substantial claim. 12

  • May combine two claims.

    Findingown data

    Most students in both groups are from engineering majors, but the ISBL group has a higher percentage of non-engineering majors (8.2%) compared to 3.1% in the control group.

    “As shown in Figure 3(a), most students in both groups are from engineering majors, but the ISBL group has a higher percentage of non-engineering majors (8.2%) compared to 3.1% in the control group (this has important implications for the results related to engineering identity as discussed later).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Two-sample t-tests for every concept/topic related to the ISBL modules indicate no significant statistical difference between the two groups related to self-assessment, while both groups report the same median for all topics.

    “Two sample t-tests are performed for every concept/topic related to the ISBL modules used.The results indicate no significant statistical difference between the two groups related to self-assessment, while both groups report the same median for all topics.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The ISBL group shows a higher level with respect to Utility compared to the control group, and the observed difference is highly statistically significant.

    “According to the test results, the ISBL group shows a higher level with respect to “Utility” compared to the control group and that the observed difference is highly statistically significant.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning

    There is a statistical difference between the two groups for Recognition; the Control group performs better with respect to this construct.

    “We observe a statistical difference between the two groups for “Recognition”; however, this time the Control group seems to be performing better with respect to this construct.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Implementing ISBL does not require any technology fee or access to special immersive technologies because the simulation software is free for academic use and the simulations can be used on any typical computer.

    “Implementing ISBL does not require any technology fee or access to special immersive technologies as the simulation software is free for academic use and the simulations can be used on any typical computer.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [25]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    The original experiential learning instrument includes two other constructs, active learning and relevance, which were excluded in this implementation due to overlap with constructs measured by the other instruments used.

    “It is worth noting that the original experiential learning instrument includes two other constructs, namely active learning and relevance, which were excluded in our implementation of this instrument due to their overlap with the constructs measured by the other instruments that we used here.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Immersive technologies provide portable and risk-free learning environments that facilitate location-independent learning.

    “Immersive technologies provide portable and risk-free learning environments that facilitate location-independent learning [18].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [18]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationcitation

    ISBL enables long-lasting development of critical thinking and problem-solving skills by activating prior knowledge, providing a contextually enriched environment, and encouraging learners to elaborate on their knowledge to solve a real-world inspired problem.

    “ISBL enables long-lasting development of critical thinking and problem-solving skills by: (a) activating relevant prior knowledge; (b) providing a contextually enriched environment (via immersive simulations) that mimics future professional settings; and, (c) encouraging learners to elaborate on their knowledge to solve a real-world inspired problem.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [21]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationcitation

    ISBL enables knowledge to be constructed via interactions with the virtual environment and indexed by relevant contexts.

    “By doing so, ISBL enables knowledge to be constructed via interactions with the virtual environment and indexed by relevant contexts.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [22]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    The improvement in Utility is because ISBL resembles real-world inspired problems, allowing students to more clearly see that what they learn is useful and applicable in real-world settings.

    “We believe this improvement is because ISBL resembles real-world inspired problems, allowing the students to more clearly see that what they learn is useful and applicable in real-world settings.”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: The two groups are comparable in terms of academic and personality factors, and any statistical difference observed between the two groups regarding the outcome variables can be attributed to the intervention (ISBL modules).

  • May combine two claims.

    Premisecitation

    Problem-based learning has proved to improve innovation, metacognition, engagement and meaningfulness.

    “PBL has proved to improve innovation [8], metacognition [9], engagement and meaningfulness [10], [11].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [8]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [9]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [10]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Problem-based learning helps students learn by applying the learned knowledge rather than memorizing it.

    “PBL helps students learn by applying the learned knowledge rather than memorizing it [15]”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [15]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Premisecitation

    Engineering economy is generally characterized as a course with a high failure rate, often attributed to engineering students' low engagement and motivation toward the topics covered.

    “Nevertheless, engineering economy is generally characterized as a course with a high failure rate, which is often attributed to engineering students' low engagement and motivation toward the topics covered in the course [6].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [6]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationreasoning

    The improvement in motivation can be explained by ISBL resembling real-world situations, leading students to see higher relevance and report a more positive attitude and confidence about handling real-world problems.

    “The improvement in motivation can be explained by noting that ISBL is inspired by and resembles real-world situations that the learner may encounter at the future workplace, hence students see higher relevance and report a more positive attitude towards success as they feel more confident about their ability to handle real-world problems.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The two groups are comparable in terms of academic and personality factors, and any statistical difference observed between the two groups regarding the outcome variables can be attributed to the intervention (ISBL modules).

  • May combine two claims.

    Interpretationown data

    Statistical comparisons from a controlled experiment conducted in an undergraduate engineering economy course show that ISBL improves motivation and experiential learning.

    “The statistical comparisons from a controlled experiment conducted in an undergraduate engineering economy course show that ISBL improves motivation and experiential learning.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The two groups are comparable in terms of academic and personality factors, and any statistical difference observed between the two groups regarding the outcome variables can be attributed to the intervention (ISBL modules).

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Problem-based learning encourages design thinking and curriculum integration.

    “In addition, it encourages design thinking [12] as well as curriculum integration [13], [14].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [12]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [13]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [14]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Study design and conductnone given

    All necessary IRB approvals were obtained prior to the experiment and data collection.

    “We use the following instruments to collect data from research participants (all necessary IRB approvals are obtained prior to the experiment and data collection).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationcitation

    ISBL aligns with Self-determination Theory by promoting autonomous motivators, unlike traditional methods that are primarily based on controlled motivators.

    “As a result, ISBL aligns with the Self-determination Theory [23] by promoting autonomous motivators, unlike traditional methods that are primarily based on controlled motivators such as rewards and punishments (e.g., passing or failing a test), which often lead to superficial learning and cause a sense of pressure and anxiety.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [23]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    In the implementations discussed, a set of traditional homework problems was replaced with related ISBL modules without restructuring or modifying other aspects of the course.

    “In the implementations discussed in this paper, we replaced a set of traditional homework problems with related ISBL modules without restructuring or modifying other aspects of the course.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Problem-based learning is recommended as an effective teaching and learning method in engineering economy courses.

    “and is recommended as an effective teaching and learning method in engineering economy courses [16].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [16]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Study design and conductown data

    The experiment involved a control and an intervention group; both groups used the same material offered by the same instructor and via the same delivery method, and the only difference was the use of the ISBL learning module instead of traditional homework assignments for the intervention group.

    “Our experiment (as described in the following section) involved a “control” and an “intervention” group.Both groups used the same material offered by the same instructor and via the same delivery method.The only difference was the use of the ISBL learning module instead of traditional homework assignments for the intervention group.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductcitation

    Interview notes were analyzed using qualitative data analysis techniques from Grounded Theory to produce a set of themes across student experiences.

    “Interview notes were taken and analyzed using qualitative data analysis techniques from Grounded Theory to produce a set of themes across student experiences [31].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [31]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationown data

    The Real World Context theme supports both the development of motivation found in hypothesis 1 and the recognition of utility found in hypothesis 2.

    “This theme supports both the development of motivation found in the assessments of hypothesis 1 and the recognition of utility found in the assessment of hypothesis 2 as a result of operating in a more real-world context.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    The immersive simulations in the ISBL modules investigated are developed using Simio simulation software, which does not incur any technology fee for academic and classroom use and is compatible with VR.

    “For the ISBL modules investigated in this paper, the immersive simulations are developed using the Simio® simulation software [25], which does not incur any technology fee for academic and classroom use and is compatible with VR, giving the learner the option to view the simulated environment on a 2D display (low-immersion mode) or via a VR headset (high-immersion mode).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [25]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Immersive technologies are shown to enhance certain learning outcomes in engineering disciplines such as creativity and spatial skills.

    “Moreover, these technologies are shown to enhance certain learning outcomes in engineering disciplines such as creativity and spatial skills [18].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [18]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationown data

    The results show that the ISBL modules enhanced motivation and experiential learning without any adverse impact on students' self-perceived learning.

    “In conclusion, the results show that the ISBL modules enhanced motivation and experiential learning without any adverse impact on students' self-perceived learning.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The two groups are comparable in terms of academic and personality factors, and any statistical difference observed between the two groups regarding the outcome variables can be attributed to the intervention (ISBL modules).
26 of 26

  • The authors state this as an assumption.

    The two groups are comparable in terms of academic and personality factors, and any statistical difference observed between the two groups regarding the outcome variables can be attributed to the intervention (ISBL modules).

    “Based on the above results, it would be reasonable to assume that the two groups are comparable in terms of academic and personality factors and that any statistical difference observed between the two groups regarding the outcome variables can be attributed to the intervention implemented, i.e., the ISBL modules.”
1 of 1

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

  • Control motivation total does not equal its component means

    Minor

    The four control-group IMMS means sum to 32.089, not the reported overall mean of 32.9. The difference is too large to be explained by rounding.

    Reported 32.9 · recomputed 32.089

1 of 1

Cites

Flagged first, then by citations

1 of the 24 references couldn’t be matched to a known paper.

Research works printed but not linked: 5

  1. D. Newnan, T. Eschenbach, J. Lavelle, and N. Lewis, Engineering Economy Analysis, 14th ed. New york: Oxford University, 2019.
  2. W. Sullivan, E. Wicks, and C. Koelling, Engineering Economy, 17th ed. Pearson, 2019.
  3. C. Park, Fundamentals of Engineering Economics, 4th ed. Pearson, 2019.
  4. A. Godwin, "The Development of a Measure of Engineering Identity American Society for Engineering Education", ASEE Annual Conference & Exposition Proceedings,2016.
  5. K. O'Reilly, Ethnographic methods. London: Routledge, 2011.

Cited by

Sorted by citations

Related

Authors
Negahban, A., Ashour, O., Nowparvar, M., Knight, D., Ozden, S., & Delgoshaei, P.
Affiliations
  • Pennsylvania State University (US)
  • University of Colorado Boulder (US)
Corresponding author
Negahban, Ashkan
ORCID
Venue
2022 ASEE Annual Conference & Exposition Proceedings
Publisher
ASEE Conferences
Published
2024-02-06
DOI
10.18260/1-2--41913
OpenAlex
W4391578128
Cited by
7
References
24
Citations by year
2024: 7
FWCI
4.79
Type
conference paper
Language
en
Open access
Locations
ASEE Annual Conference & Exposition (publishedVersion)
Funders
  • National Science Foundation (2000599)
  • University of Pennsylvania
  • Pennsylvania State University
  • Social Science Research Institute, Pennsylvania State University
Topics
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Simulation Techniques and Applications
  • Physical Sciences › Engineering › Media Technology › Experimental Learning in Engineering
  • Physical Sciences › Engineering › Architecture › Engineering Education and Pedagogy
Keywords
simulation-based training, undergraduate engineering education, agent-based modeling, discrete event simulation, learning modules

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-29.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-29.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review