The DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.
Internal, External, and Ecological Validity in Research Design, Conduct, and Evaluation
Open accessAbstract
Reliability and validity describe desirable psychometric characteristics of research instruments. The concept of validity is also applied to research studies and their findings. Internal validity examines whether the study design, conduct, and analysis answer the research questions without bias. External validity examines whether the study findings can be generalized to other contexts. Ecological validity examines, specifically, whether the study findings can be generalized to real-life settings; thus ecological validity is a subtype of external validity. These concepts are explained using examples so that readers may understand why the consideration of internal, external, and ecological validity is important for designing and conducting studies, and for understanding the merits of published research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Moher, D., Jadad, A. R., Nichol, G., Penman, M., Tugwell, P., & Walsh, S. (1995) · Controlled Clinical Trials · doi:10.1016/0197-2456(94)00031-w · Cited by 1,308
Assessing the Quality of Randomised Controlled Trials
Jüni, P., Altman, D. G., & Egger, M. (2001) · Systematic Reviews in Health Care · doi:10.1002/9780470693926.ch5 · Cited by 238
The NIMH-CATIE Schizophrenia Study: What Did We Learn?
Lieberman, J. A., & Stroup, T. S. (2011) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2011.11010039 · Cited by 113 · cited 6×
“Thus CATIE has good external validity for clinical practice in the USA but not in India.”
InfluentialThe Concept of Ecological Validity: What Are Its Limitations and Is It Bad to Be Invalid?
Lewkowicz, D. J. (2001) · Infancy · doi:10.1207/s15327078in0204_03 · Cited by 92
Andrade, C., Sudha, S., & Venkataraman, B. V. (2000) · Journal of Ect · doi:10.1097/00124509-200006000-00006 · Cited by 34 · cited 1×
“How much less ecological validity, then, would research in animal models of different neuropsychiatric states have for patients in clinical practice? This explains why drugs that work in animal models often fail in humans.[6]”
Self-citationStreiner, D. L. (2016) · Journal of Clinical Psychopharmacology · doi:10.1097/jcp.0000000000000517 · Cited by 7 · cited 1×
“Indian Journal of Psychological Medicine | Volume 40 | Issue 5 | September-October 2018 499 desirable psychometric properties of research and clinical instruments.[2,3] Validity can also be applied to laboratory and clinical studies, and to their findings, as well, as the sections below show.”
Streiner, D. L. (2016) · Journal of Clinical Psychopharmacology · doi:10.1097/jcp.0000000000000589 · Cited by 5 · cited 1×
“Indian Journal of Psychological Medicine | Volume 40 | Issue 5 | September-October 2018 499 desirable psychometric properties of research and clinical instruments.[2,3] Validity can also be applied to laboratory and clinical studies, and to their findings, as well, as the sections below show.”
Cited by
The inconvenient truth about convenience and purposive samples
Andrade, C. (2021) · Indian Journal of Psychological Medicine · doi:10.1177/0253717620977000 · Cited by 1,110
Self-citationAutomatic assessment of pain based on deep learning methods: A systematic review
Gkikas, S., & Tsiknakis, M. (2023) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2023.107365 · Cited by 164
Standardisation of quantitative resource conversion studies with black soldier fly larvae
Bosch, G., Oonincx, D. G. A. B., Jordan, H. R., Zhang, J., van Loon, J. J. A., van Huis, A., & Tomberlin, J. K. (2019) · Journal of Insects as Food and Feed · doi:10.3920/jiff2019.0004 · Cited by 153
“The design of the experiment determines the validity, which refers whether the study will answer the research questions without bias and whether the study findings can be generalized to other contexts (Andrade, 2018).”
BackgroundLyu, K., Brambilla, A., Globa, A., & de Dear, R. (2023) · Automation in Construction · doi:10.1016/j.autcon.2023.104836 · Cited by 112
Pantano, E., Viassone, M., Boardman, R., & Dennis, C. (2022) · Journal of Retailing and Consumer Services · doi:10.1016/j.jretconser.2022.103074 · Cited by 100
“This new approach would increase the reliability of the results, enhancing their ecological validity (Andrade, 2018; Araujo et al., 2007), and allowing for the potential discovery of additional factors that may have not emerged in the present research.”
Lu, D., He, Y., & Tan, Y. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.719425 · Cited by 66
“Notably, there were no any statistical considerations aiming at the ecological validity of this meta-analysis though the substantial heterogeneity for these included studies was found (Andrade, 2018).”
BackgroundShopper intent prediction from clickstream e-commerce data with minimal browsing information
Requena, B., Cassani, G., Tagliabue, J., Greco, C., & Lacasa, L. (2020) · Scientific Reports · doi:10.1038/s41598-020-73622-y · Cited by 70
The value of decentralized clinical trials: Inclusion, accessibility, and innovation
Jean‐Louis, G., & Seixas, A. A. (2024) · Science · doi:10.1126/science.adq4994 · Cited by 78
“Moreover, traditional RCTs often fail to replicate the conditions under which therapies are used in everyday clinical practice, limiting the external validity of their results ( 8 – 10 ).”
BackgroundYildirim, M., Globa, A., Göçer, Ö., & Brambilla, A. (2024) · Building and Environment · doi:10.1016/j.buildenv.2024.111841 · Cited by 67
Intent-to-Treat (ITT) vs Completer or Per-Protocol Analysis in Randomized Controlled Trials
Andrade, C. (2022) · Indian Journal of Psychological Medicine · doi:10.1177/02537176221101996 · Cited by 52
Self-citationSounds Healthy: Modelling sound-evoked consumer food choice through visual attention
Peng-Li, D., Mathiesen, S. L., Chan, R. C. K., Byrne, D. V., & Wang, Q. J. (2021) · Appetite · doi:10.1016/j.appet.2021.105264 · Cited by 52
Digital Games for Creativity Assessment: Strengths, Weaknesses and Opportunities
Rafner, J. F., Biskjær, M. M., Zana, B., Langsford, S., Bergenholtz, C., Rahimi, S., Carugati, A., Noy, L., & Sherson, J. F. (2021) · Creativity Research Journal · doi:10.1080/10400419.2021.1971447 · Cited by 64
“This is dependent on the testing environment (Andrade, 2018).”
BackgroundAndonova, V., Reinoso‐Carvalho, F., Ramírez, M. A. J., & Carrasquilla, D. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1176697 · Cited by 70
“be, in terms of applicability in real-life scenarios (Andrade, 2018).”
BackgroundSpacematch: Using Environmental Preferences to Match Occupants to Suitable Activity-Based Workspaces
Sood, T., Janssen, P. H. T., & Miller, C. (2020) · Frontiers in Built Environment · doi:10.3389/fbuil.2020.00113 · Cited by 59
A Case for Multisite Second Language Acquisition Research: Challenges, Risks, and Rewards
Moranski, K., & Ziegler, N. (2020) · Language Learning · doi:10.1111/lang.12434 · Cited by 55
Field experiments in operations management
Gao, Y. A., Li, M., & Sun, S. (2023) · Journal of Operations Management · doi:10.1002/joom.1240 · Cited by 48
Romadlon, D. S., Hasan, F., Wiratama, B. S., & Chiu, H. (2021) · Journal of Nursing Scholarship · doi:10.1111/jnu.12763 · Cited by 50
Yildirim, M., Globa, A., Göçer, Ö., & Brambilla, A. (2025) · Building and Environment · doi:10.1016/j.buildenv.2025.112608 · Cited by 43
Grindey, A., & Bradshaw, T. (2022) · International Journal of Mental Health Nursing · doi:10.1111/inm.12992 · Cited by 37
Examining explainable clinical decision support systems with think aloud protocols
Anjara, S. G., Janik, A., Dunford-Stenger, A., Mc Kenzie, K., Collazo-Lorduy, A., Torrente, M., Costabello, L., & Provencio, M. (2023) · PLOS ONE · doi:10.1371/journal.pone.0291443 · Cited by 28
López‐Mas, L., Claret, A., Bermúdez, A., Llauger, M., & Guerrero, L. (2022) · Foods · doi:10.3390/foods11091183 · Cited by 43
“Implicit techniques have long been debated on its ecological validity, which refers to whether or not the study findings in a laboratory setting can be generalised to real-life [41].”
BackgroundBarker, T. G., O’Higgins, A., Fonagy, P., & Gardner, F. E. M. (2024) · Journal of Affective Disorders · doi:10.1016/j.jad.2023.12.080 · Cited by 32
A scoping review of mathematical models covering Alzheimer's disease progression
Moravveji, S., Doyon, N., Mashreghi, J., & Duchesne, S. (2024) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2024.1281656 · Cited by 32
You, Q., Bai, D.-X., Wu, C.-X., Chen, H., Hou, C., & Gao, J. (2022) · Asian Nursing Research · doi:10.1016/j.anr.2022.11.002 · Cited by 27
Miller, B. P., Reed, D. D., & Amlung, M. T. (2023) · Journal of the Experimental Analysis of Behavior · doi:10.1002/jeab.860 · Cited by 33
Artificial intelligence in mammography: a systematic review of the external validation
Branco, P. E. S. C., Franco, A. H. S., de Oliveira, A. P., Carneiro, I. M. C., de Carvalho, L. S., de Souza, J. I. N., Leandro, D. R., & Cândido, E. B. (2024) · Revista Brasileira Ginecologia e Obstetrícia · doi:10.61622/rbgo/2024rbgo71 · Cited by 34
Ooms, A., Heaton‐Shrestha, C., Connor, S., McCawley, S., McShannon, J., Music, G., & Trainor, K. (2022) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2022.104257 · Cited by 28
“External validity was obtained through the study of the programme taking place in its natural setting (ecological validity, e.g., Andrade, 2018).”
Collecting Feature Models from the Literature: A Comprehensive Dataset for Benchmarking
Sundermann, C., Brancaccio, V. F., Kuiter, E., Krieter, S., Heß, T., & Thüm, T. (2024) · International Software Product Line Conference (SPLC) · doi:10.1145/3646548.3672590 · Cited by 34
“Usingnon-representativedatasetsforbenchmarkingperformance may lead to empirical evaluations that lack external [117] and ecological [3, 50] validity and cannot be transferred into practice [4, 42] due to several aspects.”
Sampling Methods and Sample Size Determination in Clinical Research: An Educational Review
Zrineh, A., Al‐Usta, M., & Alwawi, A. A. (2026) · Journal of General and Family Medicine · doi:10.1002/jgf2.70096 · Cited by 67
“…to the degree to which the study's findings accurately reflect the true relationships within the population being investigated [12], while external validity (generalizability) concerns the extent to which findings can be appropriately extrapolated to broader populations or different contexts [13].”
Marr, C., Breeze, P. R., & Caton, S. J. (2021) · Appetite · doi:10.1016/j.appet.2021.105777 · Cited by 23
“This allowed participants to complete the study in their own home, increasing ecological validity 38 .”
Scenarios for digital twin deployment in healthcare facilities management
Madubuike, O. C., Anumba, C. J., & Agapaki, E. (2023) · Journal of Facilities Management · doi:10.1108/jfm-10-2022-0107 · Cited by 30
Software practitioners’ point of view on technical debt payment
Freire, S., Rios, N., Pérez, B., Castellanos, C., Correal, D., Ramač, R., Mandić, V., Taušan, N., López, G., Pacheco, A., et al. (2022) · Journal of Systems and Software · doi:10.1016/j.jss.2022.111554 · Cited by 28
Reviewing the Framework of Risk Management: Policy and Hedging
Shakatreh, M., Rumman, M. A. A., & Mugableh, M. I. (2023) · International Journal of Professional Business Review · doi:10.26668/businessreview/2023.v8i1.928 · Cited by 25
“A conclusive research design is known as exploratory research design as well (Andrade, 2018).”
BackgroundThe SDE4 Learning Trail: Crowdsourcing occupant comfort feedback at a net-zero energy building
Sood, T., Quintana, M., Jayathissa, P., Abdelrahman, M., & Miller, C. C. (2019) · Journal of Physics Conference Series · doi:10.1088/1742-6596/1343/1/012141 · Cited by 21
Simões, C., Santos, A. C., Lebre, P., Daniel, J. R., Branquinho, C., Gaspar, T., & de Matos, M. G. (2021) · School Psychology International · doi:10.1177/01430343211025075 · Cited by 27
Exploring experimental research: Methodologies, designs, and applications across disciplines
Em, S. (2025) · Cambodian journal of educational and social sciences. · doi:10.69496/cjess.v2i2.78 · Cited by 32
Rouchitsas, A., & Alm, H. (2023) · Multimodal Technologies and Interaction · doi:10.3390/mti7020010 · Cited by 24
van Dellen, S. A., Wisse, B. M., & Mobach, M. P. (2022) · International Breastfeeding Journal · doi:10.1186/s13006-022-00499-0 · Cited by 29
“causal claims, because it minimizes threats to internal validity [31].”
BackgroundRouchitsas, A., & Alm, H. (2022) · Information · doi:10.3390/info13090420 · Cited by 19
Brian, A., Bostick, L., Starrett, A., Kļaviņa, A., Miedema, S. T., Pennell, A., Stribing, A., Gilbert, E. N., & Lieberman, L. J. (2020) · Adapted Physical Activity Quarterly · doi:10.1123/apaq.2019-0019 · Cited by 19
Conducting eye-tracking studies on large and interactive process models using EyeMind
Andaloussi, A. A., Lübke, D., & Weber, B. (2023) · SoftwareX · doi:10.1016/j.softx.2023.101564 · Cited by 18
AI-enabled clinical decision support tools for mental healthcare: A product review
Kleine, A., Kokje, E., Hummelsberger, P., Lermer, E., Schaffernak, I., & Gaube, S. (2024) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2024.103052 · Cited by 29
“Therefore, the second goal of the systematic review is to provide information about available scientific evidence addressing the external validity of the identified tools [29,33].”
Koedel, U., Schuetze, C., Fischer, P. F., Bussmann, I., Sauer, P. K., Nixdorf, E., Kalbacher, T., Wichert, V., Rechid, D., Bouwer, L. M., et al. (2022) · Frontiers in Environmental Science · doi:10.3389/fenvs.2021.772666 · Cited by 24
“Internal validity examines the extent to which systematic error (bias) is present (Andrade, 2018).”
BackgroundResultKondo, T., Nishigori, H., & van der Vleuten, C. P. M. (2022) · BMC Medical Education · doi:10.1186/s12909-022-03352-4 · Cited by 22
“The inability to account for local context decreases ecological validity [15], rendering the data acquired through direct use of the generic rubrics practically useless towards making summative decisions.”
Deffayet, R., Thonet, T., Renders, J.-M., & de Rijke, M. (2022) · ACM SIGIR Forum · doi:10.1145/3582900.3582905 · Cited by 29
Corti, C., Oprandi, M. C., Chevignard, M., Jansari, A. S., Oldrati, V., Ferrari, E., Martignoni, M., Romaniello, R., Strazzer, S., & Bardoni, A. (2021) · Neuropsychology Review · doi:10.1007/s11065-021-09498-0 · Cited by 21
“Ecological Validity reporting (1 = yes, 0 = no): Ecological Validity is a subtype of External Validity and indicates the extent to which the findings of a study can be generalized to naturalistic situations in everyday life (Andrade, 2018; Brunswik, 1956; Chaytor & Schmitter-Edgecombe, 2003).”
BackgroundEbenezar, U. S., Manikandan, S. P., Senthilvel, P. G., & Sivasankar, C. (2025) · Robotica · doi:10.1017/s0263574724002224 · Cited by 17
“Dai et al. [21] conducted experiments to solve these challenges by combining pain and emotion-detection datasets.”
Choi, I., & Gil-García, J. R. (2021) · International Public Management Journal · doi:10.1080/10967494.2021.1913459 · Cited by 18
“Ecological validity specifically examines the degree to which findings “ can be generalized to real-life settings, ” while external validity indicates the extent to which the results “ can be generalized to other contexts ” (Andrade 2018:499).”
BackgroundRobots for Connection: A Co-Design Study with Adolescents
Alves‐Oliveira, P., Björling, E. A., Wiesmann, P., Dwikat, H., Bhatia, S., Mihata, K., & Çakmak, M. (2022) · IEEE RO-MAN · doi:10.1109/ro-man53752.2022.9900534 · Cited by 18
“Additionally, this study was conducted in situ to ensure ecological validity [16], [17].”
MethodWhy the study of comparative psychology is important to neuroscientists
Abramson, C. I. (2023) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2022.1095033 · Cited by 18
“Internal validity evaluates if the experimental design, conduct and data analysis answer the experimental questions of a study without bias, whereas external validity refers to the extent to which the experimental finding can be generalized to a different contexts (Andrade, 2018).”
BackgroundValidity of mental and physical stress models
Hendry, E., McCallister, B., Elman, D. J., Freeman, R. L., Borsook, D., & Elman, I. (2024) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2024.105566 · Cited by 19
Pérez, J. C., Fernández, O., Cáceres, C., Carrasco, A. S., Moessner, M., Bauer, S., Espinosa-Duque, D., Gloger, S., & Krause, M. (2021) · JMIR Mental Health · doi:10.2196/26814 · Cited by 18
Diagnostic Accuracy Studies in Radiology: How to Recognize and Address Potential Sources of Bias
Pavlou, A., Kurtz, R. M., & Song, J. W. (2021) · Radiology Research and Practice · doi:10.1155/2021/5801662 · Cited by 20
“+is is especially important as it determines whether the results of the study can be applied in daily clinical practice [12].”
BackgroundPeng-Li, D., Mota, P. A. D., Corrêa, C. M. C., Chan, R. C. K., Byrne, D. V., & Wang, Q. J. (2022) · Frontiers in Neuroscience · doi:10.3389/fnins.2022.827021 · Cited by 18
“…to the nature of our controlled experimental setup, our findings cannot necessarily be directly generalized to naturalistic food choice settings (Andrade, 2018) in which multiple other external factors (including price, labeling, and social factors) may affect the consumers’ emotional states,…”
BackgroundSantiago‐Pescador, S., Fajardo‐Blanco, D., López‐Ortiz, S., Peñín‐Grandes, S., Sánchez, R. M., Lucı́a, A., Martín‐Hernández, J., & Santos‐Lozano, A. (2022) · European Journal of Sport Science · doi:10.1080/17461391.2022.2113145 · Cited by 15
“In our study, we followed an ecological validity approach (Andrade, 2018) to apply a stimulus that could be both well tolerated for participants and clinically useful.”
MethodThe logics of and strategies to enhance generalization of mixed methods research findings
Younas, A., & Durante, Á. (2023) · Methodology · doi:10.5964/meth.10863 · Cited by 26
“External validity External validity pertains to the extent to which study findings can be confidently generalized to other contexts, populations, and cases (Andrade, 2018).”
BackgroundMeurisse, M., Estupiñán‐Romero, F., González-Galindo, J., Martínez-Lizaga, N., Royo-Sierra, S., Saldner, S., Dolanski-Aghamanoukjan, L., Degelsegger-Márquez, A., Soiland‐Reyes, S., Goethem, N. V., et al. (2023) · BMC Medical Research Methodology · doi:10.1186/s12874-023-02068-3 · Cited by 20
“When this is the case, it may result in the presence of residual confounding bias in the obtained causal effect estimates, ham-pering both the internal and external validity of the study [55–57].”
BackgroundErobots as research tools: Overcoming the ethical and methodological challenges of sexology
Dubé, S., Santaguida, M., & Anctil, D. (2022) · Journal of Future Robot Life · doi:10.3233/frl-210017 · Cited by 17
“the caveat is that laboratory studies often fall short of real-world sociosexual interactions (i.e., lack of ecological validity; Andrade, 2018), whereas studies conducted in natural environments make it difficult for researchers to control experimental conditions and use sophisticated laboratory apparatus (i.”
InfluentialWhy has animal science not led to improved farm animal health and welfare?
Sundrum, A. (2024) · Frontiers in Animal Science · doi:10.3389/fanim.2023.1214889 · Cited by 15
“Reliability describes the consistency with which results are obtained (Andrade, 2018).”
Transforming Alzheimer’s Digital Caregiving through Large Language Models
Kim, S., Han, D. Y., & Bae, J. (2024) · Current Alzheimer Research · doi:10.2174/0115672050301740241118044604 · Cited by 14
Bursky, M., Kosuri, M., Walsh Carson, K., Babad, S., Iskhakova, A., & Nikulina, V. (2023) · Psychological Reports · doi:10.1177/00332941211048734 · Cited by 13
“…rigor of a controlled laboratory environment (as all of the studies were conducted in actual prison settings or quarantine or lockdown settings uncontrolled by the research teams), they do possess strong external ecological validity due to their real-world application (Andrade, 2018).”
BackgroundCoutts, K. A., & Pillay, M. (2021) · South African Journal of Communication Disorders · doi:10.4102/sajcd.v68i1.790 · Cited by 16
“It needs to be noted here that this could be a distinct advantage of using a bedside assessment over an instrumental measure, the need to assess in a realistic environment, otherwise known as ecological validity (Andrade, 2018).”
BackgroundZhang, V., Albers, A., Saeedi-Givi, C., Kristensson, P. O., Bohné, T., & Tadeja, S. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2024.3456208 · Cited by 17
“The vast majority of existing research does not thoughtfully represent real-work industrial processes [20] critical for ecological validity [4,43,50].”
BackgroundPeterson, P., Nyberg, J., Persson, C., Mark, H., & Lohmander, A. (2021) · The Cleft Palate-Craniofacial Journal · doi:10.1177/10556656211025926 · Cited by 14
“The World Health Organization (WHO, 2018) recommends adding measures of communicative activity and participation when evaluating cleft lip and palate (CLP) treatment, with the intention of increasing the ecological validity of the results (Andrade, 2018).”
BackgroundKarle, V. L., Hartung, V., Ivanovska, K., Mäurer, M., Flachenecker, P., Pfeifer, K., & Tallner, A. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17239044 · Cited by 13
Young, S. A., Newton, A. R., Fowler, S., & Park, J. Y. (2021) · Journal of Biological Education · doi:10.1080/00219266.2021.1877785 · Cited by 13
“Ecological validity also was an issue, especially with regard to novelty and Hawthorne effects (Andrade 2018).”
BackgroundLu, X., Luo, Y. G., & Hu, B. (2022) · HERD Health Environments Research & Design Journal · doi:10.1177/19375867221113067 · Cited by 12
“The field experiment provides a strong ecological validity (Andrade, 2018) by examining older adults’ movement behaviors and anxiety states in their real residential settings.”
BackgroundPoorly Recognized and Uncommonly Acknowledged Limitations of Randomized Controlled Trials
Andrade, C. (2024) · Indian Journal of Psychological Medicine · doi:10.1177/02537176241297953 · Cited by 16
Self-citationClemente, F. M., Praça, G. M., Oliveira, R. F. S., Aquino, R., Araújo, R., Silva, R., Sarmento, H., & Afonso, J. R. (2022) · International Journal of Sports Science & Coaching · doi:10.1177/17479541221085236 · Cited by 17
“the consistency with which results are obtained).(25) Moreover, when using these observational-based instruments, inter-rater reliability (i.”
Challenges for evaluation practices and innovative approaches: Lessons during COVID-19 pandemic
Srivastava, A. (2022) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2022.102095 · Cited by 16
Development and validation of an individual entrepreneurial potential new measurement scale
Ephrem, A. N., & Murimbika, M. (2023) · Journal of Research in Marketing and Entrepreneurship · doi:10.1108/jrme-07-2022-0094 · Cited by 18
Trigger warnings as tools for learning—theorising an evolving cultural concept
Nolan, H. A., & Roberts, L. M. (2023) · Medical Education · doi:10.1111/medu.15172 · Cited by 11
“34 Researchers, while discrediting a therapeutic role of warnings, concede that educators may choose to employ warnings for other reasons not investigated by them.”
BackgroundWansink, B., Timmer, J., & Bronkhorst, L. H. (2023) · Education Sciences · doi:10.3390/educsci13090938 · Cited by 18
“The cross-case design allowed for describing particularities within cases (as each classroom is unique), but also to detect meaningful patterns across cases [30].”
BackgroundThe overview effect and nature-relatedness
McKeever, N., Nezami, A., & Kourtis, D. (2024) · Frontiers in Virtual Reality · doi:10.3389/frvir.2024.1196312 · Cited by 13
“Therefore, future researchers may want to use a VR intervention for the control condition containing audio which speci fi cally addresses HNC. Ecological validity, which refers to the generalizability of results to real-world settings (Andrade, 2018), is another limitation.”
BackgroundAlsulamy, S. H., Khadar, S. D. A., Rafik, M., & Mansour, M. (2022) · Sustainability · doi:10.3390/su14095025 · Cited by 12
“To increase the internal validity of this study, we compared the data collected from interviews between researchers and between recorded notes [43].”
ResultGiordano, R., Donati, M. A., Zamboni, L., Fusina, F., Primi, C., & Lugoboni, F. (2022) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2022.854088 · Cited by 12
“Furthermore, the ecological validity of the environments guarantees greater generalizability of the skills acquired in VR to daily life environments and situations (39).”
BackgroundEvaluating and Mitigating Discrimination in Language Model Decisions
Tamkin, A., Askell, A., Lovitt, L., Durmus, E., Joseph, N., Kravec, S., Nguyen, K., Kaplan, J., & Ganguli, D. (2023) · arXiv · doi:10.48550/arxiv.2312.03689 · Cited by 11
“An inherent challenge of evaluations is external validity : ensuring the conclusions of a research study generalize to real-world settings (Andrade, 2018).”
BackgroundInformation processing on smartphones in public versus private
Ohme, J., Searles, K., & de Vreese, C. H. (2022) · Journal of Computer-Mediated Communication · doi:10.1093/jcmc/zmac022 · Cited by 12
“This triangulation promises optimal insights into cognitive mechanisms, such as attention and learning, essential to understanding the unique challenges of today’s information environment (Andrade, 2018).”
BackgroundSiette, J., Guion, J., Ijaz, K., Strutt, P. A., Porte, M., Savage, G. R., & Richards, D. (2024) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-024-02478-3 · Cited by 11
“The Leaf Café module is modelled on a natural setting, providing increased ecological validity, which improves the generalisation of findings to real life [45, 46].”
BackgroundTowards an ecologically valid naturalistic cognitive neuroscience of memory and event cognition
Pooja, R., Ghosh, P., & Sreekumar, V. (2024) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2024.108970 · Cited by 18
Reporting checklists in neuroimaging: promoting transparency, replicability, and reproducibility
Ekhtiari, H., Zare-Bidoky, M., Sangchooli, A., Valyan, A., Abi‐Dargham, A., Cannon, D. M., Carter, C. S., Garavan, H., George, T. P., Ghobadi‐Azbari, P., et al. (2024) · Neuropsychopharmacology · doi:10.1038/s41386-024-01973-5 · Cited by 15
Peer influence on eating behaviour in early childhood: A scoping review
Street, S., Simoncini, K., & Byrne, R. (2024) · Appetite · doi:10.1016/j.appet.2024.107708 · Cited by 17
“Such experimental findings may not be ecologically valid (Andrade, 2018), for example restricting the eating window to 60 s does not reflect a typical eating occasion (DeJesus et al., 2018).”
BackgroundLi, D., Mao, W., Pereira, F. C., Yu, X., Su, X. J., & Krueger, R. (2025) · Transportation Research Part F Traffic Psychology and Behaviour · doi:10.1016/j.trf.2025.05.031 · Cited by 14
Hart, M. J., Sung, J. Y., McQuillin, S. D., & Schleider, J. L. (2022) · Journal of Community Psychology · doi:10.1002/jcop.22927 · Cited by 14
Hirschhorn, R., Biderman, D., Biderman, N., Yaron, I., Bennet, R., Plotnik, M., & Mudrik, L. (2024) · Behavior Research Methods · doi:10.3758/s13428-024-02401-8 · Cited by 14
“This further raises the concern that the results might not generalize to real-life settings and circumstances (affecting their ecological validity: Andrade, 2018, as well as their overall external validity: Bracht & Glass, 1968).”
BackgroundBrouqui, P., Million, M., Parola, P., McCullough, P. A., & Raoult, D. (2023) · New Microbes and New Infections · doi:10.1016/j.nmni.2023.101188 · Cited by 9
“The monocentric nature of the study might limit the generalizability of the findings, but this design might also reinforce internal validity [11,19].”
BackgroundRhythm and Reading: Connecting the Training of Musical Rhythm to the Development of Literacy Skills
Ahokas, J. R., Saarikallio, S. H., Welch, G. F., Parviainen, T. M., & Louhivuori, J. (2024) · Early Childhood Education Journal · doi:10.1007/s10643-024-01654-4 · Cited by 18
“Ecological validity is a high standard goal for any sci - entific work (Andrade, 2018 ).”
The challenge of ecological validity in temporomandibular disorders research
Nickel, J. C., Gonzalez, Y. M., Gallo, L. M., & Iwasaki, L. R. (2024) · Journal of Oral Rehabilitation · doi:10.1111/joor.13707 · Cited by 9
Dippold, D. (2024) · Applied Linguistics · doi:10.1093/applin/amae033 · Cited by 12
Mustafovska, J., Wilson, P. H., Cole, M. H., & McGuckian, T. B. (2025) · Gait & Posture · doi:10.1016/j.gaitpost.2025.04.012 · Cited by 13
“It is also ideal that these dual-task paradigms are representative of real-world contexts; dual-task paradigms that resemble the demands and sometimes challenges of the real-world, are perceived as more reliable in informing clinical practice [7] and the design of dual-task training programs [8].”
The impact of chronotype on circadian rest-activity rhythm and sleep characteristics across the week
Brooks, C., Kabiri, N. S., Bhangu, J. S., Cai, X., Pickering, E. H., Erb, M. K., Auerbach, S. H., Bonato, P., Moore, T. L., Mortazavi, F., et al. (2021) · Chronobiology International · doi:10.1080/07420528.2021.1937197 · Cited by 13
“In other words, wellcontrolled in-laboratory studies lack “ecological validity”; they offer unparalleled resolution and specificity, yet this narrow scope inherently limits their generalizability and therefore our ability to translate their findings into clinical and functional applications (Andrade 2018; Roenneberg e…”
BackgroundOlsen, P. Ø., Tully, M. A., Cruz, B. D. P., Wegner, M., & Caserotti, P. (2022) · Age and Ageing · doi:10.1093/ageing/afac137 · Cited by 10
Reddy, K. S., Taksande, A. M., & Kurian, B. (2024) · Cureus · doi:10.7759/cureus.55004 · Cited by 17
McCoy, J., Nielsen, S., & Bruno, R. (2021) · Addiction · doi:10.1111/add.15683 · Cited by 10
Nagendrababu, V., Abbott, P. V., Boutsioukis, C., Duncan, H. F., Faggion, C. M., Kishen, A., Murray, P. E., Pulikkotil, S. J., & Dummer, P. M. H. (2022) · International Endodontic Journal · doi:10.1111/iej.13682 · Cited by 12
“Whereas the external validity of a study describes the ability of the results to be generalized for all similar studies (Andrade, 2018).”
Karpowicz, B., Masłyk, R., Skorupska, K., Jabłoński, D., Kalinowski, K., Kobyliński, P., Pochwatko, G., Kornacka, M., & Kopeć, W. (2022) · Lecture notes in networks and systems · doi:10.1007/978-3-031-11432-8_23 · Cited by 9
Borba, J. V., Canzian, J., Resmim, C. M., da Silva, R. M., Duarte, M. C. F., Mohammed, K. A., Schoenau, W., Adedara, I. A., & Rosemberg, D. B. (2024) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2024.105715 · Cited by 10
Yildirim, M., Globa, A., Göçer, Ö., & Brambilla, A. (2025) · Virtual Reality & Intelligent Hardware · doi:10.1016/j.vrih.2025.10.001 · Cited by 16
Campagnini, S., Pasquini, G., Schlechtriem, F., Fransvea, G., Simoni, L., Gerli, F., Magaldi, F., Cristella, G., Riener, R., Carrozza, M. C., et al. (2023) · Sensors · doi:10.3390/s23136059 · Cited by 8
“To overcome these drawbacks, according to the modern concept of ecological validity [14], Photocell Array devices (PA) have been introduced, allowing for the quantification of spatiotemporal gait parameters in less structured environments [15].”
BackgroundReal World Studies: What They Are and What They Are Not
Andrade, C. (2023) · Indian Journal of Psychological Medicine · doi:10.1177/02537176231188563 · Cited by 8
Self-citation
Metadata
- Authors
- Andrade, C.
- Affiliations
- National Institute of Mental Health and Neurosciences (IN)
- Corresponding author
- Andrade, Chittaranjan
- Venue
- Indian Journal of Psychological Medicine
- Publisher
- SAGE Publications
- Published
- 2018-09
- Volume
- 40
- Issue
- 5
- Pages
- 498-499
- ISSN
- 0253-7176 (ISSN-L)
- 0975-1564
- PMC
- PMC6149308
- OpenAlex
- W2890570810
- Cited by
- 321
- References
- 7
- Citations by year
- 2026: 29 · 2025: 62 · 2024: 66 · 2023: 72 · 2022: 45 · 2021: 30 · 2020: 10 · 2019: 6 · 2018: 1
- FWCI
- 38.18
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- Locations
- Indian Journal of Psychological Medicine (publishedVersion, cc-by-nc-sa)
- Europe PMC (PubMed Central) (publishedVersion, cc-by-nc-sa)
- PubMed Central (publishedVersion, cc-by-nc-sa)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Social Sciences › Education › Diverse Academic Research Areas
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Keywords
- internal validity, generalizability, ecological validity, research design, research bias, psychometric validity, research evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.