A Framework for the Testing and Validation of Simulated Environments in Experimentation and Training
Open accessThe DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
New computer technologies, like virtual reality (VR), have created opportunities to study human behavior and train skills in novel ways. VR holds significant promise for maximizing the efficiency and effectiveness of skill learning in a variety of settings (e.g., sport, medicine, safety-critical industries) through immersive learning and augmentation of existing training methods. In many cases the adoption of VR for training has, however, preceded rigorous testing and validation of the simulation tool. In order for VR to be implemented successfully for both training and psychological experimentation it is necessary to first establish whether the simulation captures fundamental features of the real task and environment, and elicits realistic behaviors. Unfortunately evaluation of VR environments too often confuses presentation and function, and relies on superficial visual features that are not the key determinants of successful training outcomes. Therefore evidence-based methods of establishing the fidelity and validity of VR environments are required. To this end, we outline a taxonomy of the subtypes of fidelity and validity, and propose a variety of practical methods for testing and validating VR training simulations. Ultimately, a successful VR environment is one that enables transfer of learning to the real-world. We propose that key elements of psychological, affective and ergonomic fidelity, are the real determinants of successful transfer. By adopting an evidence-based approach to VR simulation design and testing it is possible to develop valid environments that allow the potential of VR training to be maximized.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments
Slater, M. (2009) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2009.0138 · Cited by 2,691
When and where do we apply what we learn?: A taxonomy for far transfer
Barnett, S. M., & Ceci, S. J. (2002) · Psychological Bulletin · doi:10.1037/0033-2909.128.4.612 · Cited by 2,076
Measuring and defining the experience of immersion in games
Jennett, C., Cox, A. L., Cairns, P., Dhoparee, S., Epps, A., Tijs, T., & Walton, A. (2008) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2008.04.004 · Cited by 2,065 · cited 1×
“, 2000) or online using psychophysiological indices like eye-movements, electroencephalography (EEG) and heart rate (Jennett et al., 2008; Nacke and Lindley, 2008).”
BackgroundEnhancing Our Lives with Immersive Virtual Reality
Slater, M., & Sanchez-Vives, M. V. (2016) · Frontiers in Robotics and AI · doi:10.3389/frobt.2016.00074 · Cited by 1,993
Kraiger, K., Ford, J. K., & Salas, E. (1993) · Journal of Applied Psychology · doi:10.1037/0021-9010.78.2.311 · Cited by 1,831
A review of the use of virtual reality head-mounted displays in education and training
Jensen, L., & Konradsen, F. (2018) · Education and Information Technologies · doi:10.1007/s10639-017-9676-0 · Cited by 1,638 · cited 3×
“Important elements of instructional design should be considered to maximize the impact of simulation based training, such as levels of complexity and specificity in learning objectives, scaffolding of learning and evaluation of training (Kirschner, 2002; Carruth, 2017; Jensen and Konradsen, 2018).”
BackgroundTransfer of Training: A Meta-Analytic Review
Blume, B. D., Ford, J. K., Baldwin, T. T., & Huang, J. L. (2010) · Journal of Management · doi:10.1177/0149206309352880 · Cited by 1,477 · cited 1×
“Meanwhile practical and pedagogical factors will have both a direct and moderating effect on training outcomes (Blume et al., 2010).”
BackgroundVirtual Reality: How Much Immersion Is Enough?
Bowman, D. A., & McMahan, R. P. (2007) · Computer · doi:10.1109/mc.2007.257 · Cited by 1,375
The influence of improvement in one mental function upon the efficiency of other functions. (I).
Woodworth, R. S., & Thorndike, E. L. (1901) · Psychological Review · doi:10.1037/h0074898 · Cited by 1,223
Cognitive load theory: implications of cognitive load theory on the design of learning
Kirschner, P. A. (2002) · Learning and Instruction · doi:10.1016/s0959-4752(01)00014-7 · Cited by 917 · cited 3×
“Important elements of instructional design should be considered to maximize the impact of simulation based training, such as levels of complexity and specificity in learning objectives, scaffolding of learning and evaluation of training (Kirschner, 2002; Carruth, 2017; Jensen and Konradsen, 2018).”
BackgroundMorina, N., Ijntema, H., Meyerbröker, K., & Emmelkamp, P. M. G. (2015) · Behaviour Research and Therapy · doi:10.1016/j.brat.2015.08.010 · Cited by 439
Does Far Transfer Exist? Negative Evidence From Chess, Music, and Working Memory Training
Sala, G., & Gobet, F. (2017) · Current Directions in Psychological Science · doi:10.1177/0963721417712760 · Cited by 309 · cited 2×
“Achieving far transfer of learning that is generalized across domains which are only loosely related to each other is, however, notoriously difficult to achieve (Sala and Gobet, 2017).”
BackgroundBotella, C., Serrano, B., baños, R., & García-Palacios, A. (2015) · Neuropsychiatric Disease and Treatment · doi:10.2147/ndt.s89542 · Cited by 223
Virtual Reality Applications for Stress Management Training in the Military
Pallavicini, F., Argenton, L., Toniazzi, N., Aceti, L., & Mantovani, F. (2016) · Aerospace Medicine and Human Performance · doi:10.3357/amhp.4596.2016 · Cited by 182 · cited 1×
“Hence, VR environments capable of eliciting some degree of emotional response may have significant benefits for preparing performers for pressurized environments (Pallavicini et al., 2016).”
BackgroundĆosić, K., Popović, S., Kukolja, D., Horvat, M., & Dropuljić, B. (2010) · Cyberpsychology Behavior and Social Networking · doi:10.1089/cyber.2009.0260 · Cited by 62
Cited by
Virtual reality simulations in nurse education: A systematic mapping review
Plotzky, C., Lindwedel, U., Sorber, M., Loessl, B., König, P., Kunze, C., Kugler, C., & Meng, M. (2021) · Nurse Education Today · doi:10.1016/j.nedt.2021.104868 · Cited by 362
Metadata
- Authors
- Harris, D. J., Bird, J. M., Smart, P. A., Wilson, M. R., & Vine, S. J.
- Venue
- Frontiers in Psychology
- Publisher
- Frontiers Media SA
- Published
- 2020-03-31
- Volume
- 11
- Pages
- 605-605
- ISSN
- 1664-1078 (ISSN-L)
- OpenAlex
- W3013718853
- Cited by
- 202
- References
- 15
- FWCI
- 14.71
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.