A Psychometric Toolbox for Testing Validity and Reliability

DeVon, Holli A.; Block, Michelle E.; Moyle‐Wright, Patricia; Ernst, Diane M.; Hayden, Susan J.; Lazzara, Deborah J.; Savoy, Suzanne M.; Kostas‐Polston, Elizabeth

doi:10.1111/j.1547-5069.2007.00161.x

Cited by 1,889Health Education and ValidationPsychometric Methodologies and TestingReliability and Agreement in Measurement

A Psychometric Toolbox for Testing Validity and Reliability

DeVon, Holli A., Block, Michelle E., Moyle‐Wright, Patricia, Ernst, Diane M., Hayden, Susan J., Lazzara, Deborah J., Savoy, Suzanne M., Kostas‐Polston, Elizabeth·2007·Journal of Nursing Scholarship

DOI 10.1111/j.1547-5069.2007.00161.x

Open this paper in CoChat — free account

Abstract

Purpose To review the concepts of reliability and validity, provide examples of how the concepts have been used in nursing research, provide guidance for improving the psychometric soundness of instruments, and report suggestions from editors of nursing journals for incorporating psychometric data into manuscripts. Methods CINAHL, MEDLINE, and PsycINFO databases were searched using key words: validity, reliability, and psychometrics. Nursing research articles were eligible for inclusion if they were published in the last 5 years, quantitative methods were used, and statistical evidence of psychometric properties were reported. Reports of strong psychometric properties of instruments were identified as well as those with little supporting evidence of psychometric soundness. Findings Reports frequently indicated content validity but sometimes the studies had fewer than five experts for review. Criterion validity was rarely reported and errors in the measurement of the criterion were identified. Construct validity remains underreported. Most reports indicated internal consistency reliability (alpha) but few reports included reliability testing for stability. When retest reliability was asserted, time intervals and correlations were frequently not included. Conclusions Planning for psychometric testing through design and reducing nonrandom error in measurement will add to the reliability and validity of instruments and increase the strength of study findings. Underreporting of va…

Integrity signals

Registry

Crossref · 10.1111/j.1547-5069.2007.00161.x

Cites

0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Survey questions were derived from two previous studies on the topic, contributing to both content and face validity.
“Questions were derived from the two previous studies on the topic (Wilson et al. 2010(Wilson et al. , 2013)), thus contributing to both content and face validity (DeVon et al. 2007).”
From Understanding skill acquisition among registered nurses: the ‘perpetual novice’ phenomenon (2015) · cites this paper as DeVon et al. 2007

Construct validity in psychological tests.
Cronbach, L. J., & Meehl, P. E. (1955) · Psychological Bulletin · doi:10.1037/h0040957 · Cited by 11,195 · see citing works
The content validity index: Are you sure you know what's being reported? Critique and recommendations
Polit, D. F., & Beck, C. T. (2006) · Research in Nursing & Health · doi:10.1002/nur.20147 · Cited by 7,510 · cited 2× · see citing works
“The published literature, including texts and research articles, include extensive definitions of validity and reliability, though many pertain to comprehensive exploration of a single psychometric concept (Polit & Beck, 2006).”
BackgroundMethod
A QUANTITATIVE APPROACH TO CONTENT VALIDITY<sup>1</sup>
LAWSHE, C. H. (1975) · Personnel Psychology · doi:10.1111/j.1744-6570.1975.tb01393.x · Cited by 7,313 · see citing works
Reliability and Validity Assessment
Carmines, E., & Zeller, R. (1979) · doi:10.4135/9781412985642 · Cited by 5,884 · see citing works
Advanced Design in Nursing Research
Brink, P., & Wood, M. (1998) · doi:10.4135/9781452204840 · Cited by 433 · see citing works

Flagged first, then by citations

Sorted by citations

The Delphi Technique in Nursing and Health Research
Keeney, S., Hasson, F., & McKenna, H. (2011) · Wiley eBooks · doi:10.1002/9781444392029 · Cited by 1,568 · see citing works
Design and Implementation Content Validity Study: Development of an instrument for measuring Patient-Centered Communication
Zamanzadeh, V., Ghahramanian, A., Rassouli, M., Abbaszadeh, A., Alavi-Majd, H., & Nikanfar, A.-R. (2015) · Journal of Caring Sciences · doi:10.15171/jcs.2015.017 · Cited by 1,540 · see citing works
“174 | Journal of Caring Sciences, Jun 2015; 4 (2), 165-178 Copyright © 2015 by Tabriz University of Medical Sciences Table 4.”
BackgroundMethodInfluential
Evaluation of methods used for estimating content validity
Almanasreh, E., Moles, R. J., & Chen, T. F. (2018) · Research in Social and Administrative Pharmacy · doi:10.1016/j.sapharm.2018.03.066 · Cited by 1,550 · see citing works
Validade de conteúdo nos processos de construção e adaptação de instrumentos de medidas
Alexandre, N. M. C., & Coluci, M. Z. O. (2011) · Ciência & Saúde Coletiva · doi:10.1590/s1413-81232011000800006 · Cited by 1,354 · see citing works
Principles and methods of validity and reliability testing of questionnaires used in social and health science researches
Bolarinwa, O. A. (2015) · Nigerian Postgraduate Medical Journal · doi:10.4103/1117-1936.173959 · Cited by 1,213 · see citing works
Enhancing rigour in the Delphi technique research
Hasson, F., & Keeney, S. (2011) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2011.04.005 · Cited by 771 · see citing works
A Brief Instrument to Assess Both Burnout and Professional Fulfillment in Physicians: Reliability and Validity, Including Correlation with Self-Reported Medical Errors, in a Sample of Resident and Practicing Physicians
Trockel, M., Bohman, B., Lesure, E., Hamidi, M. S., Welle, D., Roberts, L., & Shanafelt, T. (2018) · Academic Psychiatry · doi:10.1007/s40596-017-0849-3 · Cited by 602 · see citing works
Construção de instrumentos de medida na área da saúde
Coluci, M. Z. O., Alexandre, N. M. C., & Milani, D. (2015) · Ciência & Saúde Coletiva · doi:10.1590/1413-81232015203.04332013 · Cited by 563 · see citing works
Examining patterns of adversity in Chinese young adults using the Adverse Childhood Experiences-International Questionnaire (ACE-IQ).
Ho, G., Chan, A. C. Y., Chien, W. T., Bressington, D., & Karatzias, T. (2019) · International Journal of Child Abuse & Neglect · doi:10.1016/j.chiabu.2018.11.009 · Cited by 426 · see citing works
Investigating quality of life and health-related quality of life in infertility: a systematic review
Chachamovich, J. R., Chachamovich, E., Ezer, H., Fleck, M. P., Knauth, D., & Passos, E. P. (2010) · Journal of Psychosomatic Obstetrics & Gynecology · doi:10.3109/0167482x.2010.481337 · Cited by 372 · see citing works
Establishing Validity of the Consensus Auditory-Perceptual Evaluation of Voice (CAPE-V)
Zraick, R. I., Kempster, G. B., Connor, N. P., Thibeault, S. L., Klaben, B. K., Bursac, Z., Thrush, C. R., & Glaze, L. E. (2011) · American Journal of Speech-Language Pathology · doi:10.1044/1058-0360(2010/09-0105 · Cited by 300 · see citing works
Making Sense of Methods and Measurement: Lawshe's Content Validity Index
Gilbert, G. E., & Prion, S. K. (2016) · Clinical Simulation in Nursing · doi:10.1016/j.ecns.2016.08.002 · Cited by 309 · see citing works
Reliability and validity in a nutshell
Bannigan, K., & Watson, R. A. (2009) · Journal of Clinical Nursing · doi:10.1111/j.1365-2702.2009.02939.x · Cited by 288 · see citing works
Executive Function in Autism Spectrum Disorder: History, Theoretical Models, Empirical Findings, and Potential as an Endophenotype
Demetriou, E. A., DeMayo, M. M., & Guastella, A. J. (2019) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2019.00753 · Cited by 274 · see citing works
“Validity refers to the extent that the EF assessment tool accurately taps the theorized EF construct (36, 114).”
Background
Physical activity and suicidal ideation: A systematic review and meta-analysis
Vancampfort, D., Hallgren, M., Firth, J., Rosenbaum, S., Schuch, F. B., Mugisha, J., Probst, M., Van Damme, T., Carvalho, A. F., & Stubbs, B. (2018) · Journal of Affective Disorders · doi:10.1016/j.jad.2017.08.070 · Cited by 255 · see citing works
“The acceptability of the psychometric properties of measurement tools was assessed according to previous recommendations (DeVon et al., 2007).”
Method
Screening Children for Social Determinants of Health: A Systematic Review
Sokol, R. L., Austin, A. E., Chandler, C., Byrum, E., Bousquette, J., Lancaster, C., Doss, G., Dotson, A. B., Urbaeva, V., Singichetti, B., et al. (2019) · PEDIATRICS · doi:10.1542/peds.2019-1622 · Cited by 282 · see citing works
“Authors of studies in this review developed and/or used a tool to screen children and youth for SDOHs.”
BackgroundInfluential
Developing and Validating a Questionnaire to Measure Spirituality: A Psychometric Process
Parsian, N., & Dunning, T. (2009) · Global Journal of Health Science · doi:10.5539/gjhs.v1n1p2 · Cited by 204 · see citing works
“…reliability: both were used to examine the reliability of the SQ. 2.3.1 Internal Consistency Reliability Internal consistency examines the inter-item correlations within an instrument and indicates how well the items fit together conceptually (Nunnally & Bernstein 1994; DeVon et al. 2007).”
BackgroundMethodInfluential
Nurse Competence Scale: a systematic and psychometric review
Flinkman, M., Leino‐Kilpi, H., Numminen, O., Jeon, Y., Kuokkanen, L., & Meretoja, R. (2016) · Journal of Advanced Nursing · doi:10.1111/jan.13183 · Cited by 220 · see citing works
“Acceptable range of CVI depends on the number of experts evaluating the scale (DeVon et al. 2007).”
Objective Structured Assessment of Debriefing
Arora, S., Ahmed, M., Paige, J. T., Nestel, D., Runnacles, J., Hull, L., Darzi, A. W., & Sevdalis, N. (2012) · Annals of Surgery · doi:10.1097/sla.0b013e3182610c91 · Cited by 206 · see citing works
The quiet quitting scale: Development and initial validation
Galanis, P., Katsiroumpa, A., Vraka, I. S., Siskou, O. C., Konstantakopoulou, O., Moisoglou, I., Gallos, P., & Καϊτελίδου, Δ. (2023) · AIMS Public Health · doi:10.3934/publichealth.2023055 · Cited by 166 · see citing works
“Acceptable values for inter-item correlation ranged from 0.15 to 0.75 [39] and ≥0.30 for item-total correlation [40].”
BackgroundMethodInfluential
Domains of teacher identity: A review of quantitative measurement instruments
Hanna, F., Oostdam, R., Severiens, S. E., & Zijlstra, B. J. H. (2019) · Educational Research Review · doi:10.1016/j.edurev.2019.01.003 · Cited by 174 · see citing works
Stress and burnout in health care workers during COVID-19 pandemic: validation of a questionnaire
Talaee, N., Varahram, M., Jamaati, H., Salimi, A., Attarchi, M., Dizaji, M. K., Sadr, M., Hassani, S., Farzanegan, B., Monjazebi, F., et al. (2020) · Journal of Public Health · doi:10.1007/s10389-020-01313-z · Cited by 208 · see citing works
Managing in turbulence: how the capacity for resilience influences creativity
Richtnér, A. D., & Löfsten, H. (2014) · R and D Management · doi:10.1111/radm.12050 · Cited by 187 · see citing works
The childbirth experience: A systematic review of predictors and outcomes
McKelvin, G., Thomson, P. G., & Downe, S. M. (2020) · Women and Birth · doi:10.1016/j.wombi.2020.09.021 · Cited by 179 · see citing works
Translation, Cross-Cultural Adaptation, and Validation of the Malay Version of the System Usability Scale Questionnaire for the Assessment of Mobile Apps
Marzuki, M. F. M., Yaacob, N. A., & Yaacob, N. M. (2018) · JMIR Human Factors · doi:10.2196/10308 · Cited by 161 · see citing works
“A higher alpha value indicates a higher internal reliability of the questionnaire and value more than .70 is acceptable as satisfactory internal reliability [16].”
What Do You Think You Are Measuring? A Mixed-Methods Procedure for Assessing the Content Validity of Test Items and Theory-Based Scaling
Koller, I., Levenson, M. R., & Glück, J. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.00126 · Cited by 169 · see citing works
“Currently, content validity is rarely mentioned in psychological journals but receives special attention in other disciplines such as nursing research (e.g., de Von et al., 2007).”
BackgroundInfluential
Best Practices for Conducting and Interpreting Studies to Validate Self-Report Dietary Assessment Methods
Kirkpatrick, S. I., Baranowski, T., Subar, A. F., Tooze, J. A., & Frongillo, E. A. (2019) · Journal of the Academy of Nutrition and Dietetics · doi:10.1016/j.jand.2019.06.010 · Cited by 180 · see citing works
“Content validity is also subjective but relies upon experts and feedback mechanisms (such as the Delphi technique(88)) to develop and evaluate items to capture the salient features or dimensions of a construct.(35) For example, although a lay person may be able to list vegetables that should be”
BackgroundMethodInfluential
Spanish Version of the System Usability Scale for the Assessment of Electronic Tools: Development and Validation
Sevilla-González, M. D. R., Loaeza, L. M., Lazaro-Carrera, L. S., Bourguet-Ramirez, B., Rodríguez, A. V., Peralta‐Pedrero, M. L., & Almeda‐Valdés, P. (2020) · JMIR Human Factors · doi:10.2196/21161 · Cited by 158 · see citing works
Quantification of risk perception: Development and validation of the construction worker risk perception (CoWoRP) scale
Man, S. S., Chan, A. H. S., & Alabdulkarim, S. A. (2019) · Journal of Safety Research · doi:10.1016/j.jsr.2019.09.009 · Cited by 141 · see citing works
Debriefing Experience Scale: Development of a Tool to Evaluate the Student Learning Experience in Debriefing
Reed, S. J. (2012) · Clinical Simulation in Nursing · doi:10.1016/j.ecns.2011.11.002 · Cited by 131 · see citing works
“Testeretest reliability, administering the same test to the same group of respondents at different times, could also be used, in addition to Cronbach’s alpha coefficient calculations to establish reliability of the scale (DeVon et al., 2007).”
Influential
Aspirations: An Approach to Measurement with Validation Using Ethiopian Data
Bernard, T., & Taffesse, A. S. (2014) · Journal of African Economies · doi:10.1093/jae/ejt030 · Cited by 132 · see citing works
Practical Guidelines to Develop and Evaluate a Questionnaire
Kishore, K., Jaswal, V., Kulkarni, V., & De, D. (2021) · Indian Dermatology Online Journal · doi:10.4103/idoj.idoj_674_20 · Cited by 174 · see citing works
“PA is the most robust empirical technique to retain the appropriate number of factors.[16,20] However, it should be used cautiously for the eigenvalue near the 95th percentile cutoff line.”
Refining the conceptualization of Brand Authenticity
Akbar, M. M., & Wymer, W. (2017) · Journal of Brand Management · doi:10.1057/s41262-016-0023-3 · Cited by 138 · see citing works
Reliability and validity of student peer assessment in medical education: A systematic review
Speyer, R., Pilz, W., Van Der Kruis, J., & Brunings, J. W. (2011) · Medical Teacher · doi:10.3109/0142159x.2011.610835 · Cited by 123 · see citing works
“2006a) the extent to which different measures of the same construct correlate with one other (DeVon et al. 2007).”
Background
More Than a Tick Box: Medical Checklist Development, Design, and Use
Burian, B. K., Clebone, A., Dismukes, K., & Ruskin, K. J. (2018) · Anesthesia & Analgesia · doi:10.1213/ane.0000000000002286 · Cited by 133 · see citing works
The Language Exposure Assessment Tool: Quantifying Language Exposure in Infants and Children
Anda, S. D., Bosch, L., Poulin‐Dubois, D., Zesiger, P., & Friend, M. (2016) · Journal of Speech Language and Hearing Research · doi:10.1044/2016_jslhr-l-15-0234 · Cited by 153 · see citing works
“Criterion validity refers to the relation between a test and performance on another theoretically related measure (DeVon et al., 2007; Waltz, 2005; Woehr & Arthur, 2003).”
Background
Factors Influencing Consumer Behavior and Prospective Purchase Decisions in a Dynamic Pricing Environment — An Exploratory Factor Analysis Approach
Victor, V., Thoppan, J. J., Nathan, R. J., & Maria, F. F. (2018) · Social Science Research Network · doi:10.3390/socsci7090153 · Cited by 118
“Internal consistency checks the correlation within the items in an instrument, and shows how well the given items fit to a conceptual model (Nunnally and Bernstein 1994; Devon et al. 2007).”
Systems Measures of a Trauma-Informed Approach: A Systematic Review
Champine, R. B., Lang, J. M., Nelson, A., Hanson, R. F., & Tebes, J. (2019) · American Journal of Community Psychology · doi:10.1002/ajcp.12388 · Cited by 116 · see citing works
The Effect of the Hidden Curriculum on Resident Burnout and Cynicism
Billings, M. E., Lazarus, M. E., Wenrich, M. D., Curtis, J. R., & Engelberg, R. A. (2011) · Journal of Graduate Medical Education · doi:10.4300/jgme-d-11-00044.1 · Cited by 120 · see citing works
Development and Validation of a Tool to Measure Self-Confidence and Anxiety in Nursing Students During Clinical Decision Making
White, K. A. (2014) · Journal of Nursing Education · doi:10.3928/01484834-20131118-05 · Cited by 124 · see citing works
Assessing the professional identity of primary student teachers: Design and validation of the Teacher Identity Measurement Scale
Hanna, F., Oostdam, R., Severiens, S. E., & Zijlstra, B. J. H. (2020) · Studies in Educational Evaluation · doi:10.1016/j.stueduc.2019.100822 · Cited by 138 · see citing works
Development and evaluation of the content validity, practicability and feasibility of the Innovative dementia-oriented Assessment system for challenging behaviour in residents with dementia
Halek, M., Holle, D., & Bartholomeyczik, S. (2017) · BMC Health Services Research · doi:10.1186/s12913-017-2469-8 · Cited by 126 · see citing works
“However, despite its limited value within the validity hierarchy [50, 54], content validity is an important quality indicator of an instrument’s validity and provides insight into its feasibility and practicability [32, 51, 54].”
Background
Tutorial on how to calculating content validity of scales in medical research
Madadizadeh, F., & Bahariniya, S. (2023) · Perioperative Care and Operating Room Management · doi:10.1016/j.pcorm.2023.100315 · Cited by 97 · see citing works
Review Article: A Systematic Review of Cancer-Related Fatigue Measurement Questionnaires
Seyidova-Khoshknabi, D., Davis, M. P., & Walsh, D. (2010) · American Journal of Hospice and Palliative Medicine® · doi:10.1177/1049909110381590 · Cited by 108 · see citing works
“9 by Cronbach a coefficient.(53) In our estimation, the tool that has the greatest utility for those with advanced cancer undergoing palliative therapies would be the BFI since it has less than 10 items and optimal psychometric properties.”
Method
Validation of the UNESP-Botucatu unidimensional composite pain scale for assessing postoperative pain in cattle
de Oliveira, F. A., Luna, S. P. L., do Amaral, J. B., Rodrigues, K. A., Sant’Anna, A. C., Daolio, M., & Brondani, J. T. (2014) · BMC Veterinary Research · doi:10.1186/s12917-014-0200-0 · Cited by 116 · see citing works
“According to the Cronbach's α value, the scale employed in this work has excellent internal consistency [45,60].”
Method
Content Validity and Reliability of the Measurement Tools in Educational, Behavioral, and Health Sciences Research
Vakili, M. M., & Jahangiri, N. (2018) · Journal of Medical Education Development · doi:10.29252/edcj.10.28.106 · Cited by 107 · see citing works
The Virtual Shop: A new immersive virtual reality environment and scenario for the assessment of everyday memory
Ouellet, É., Boller, B., Corriveau‐Lecavalier, N., Cloutier, S., & Belleville, S. (2018) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2018.03.010 · Cited by 108 · see citing works
The development of the acceptance and commitment therapy fidelity measure (ACT-FM): a delphi study and field test
O'Neill, L., Latchford, G., McCracken, L. M., & Graham, C. D. (2019) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2019.08.008 · Cited by 87 · see citing works
“While Delphi Studies can be used to enhance the emergent face or construct validity of a measure (DeVon et al., 2007; Hardesty & Bearden, 2004; e.g., Salava ti et al., 2017), they”
Method
Physical activity correlates in people living with HIV/AIDS: a systematic review of 45 studies
Vancampfort, D., Mugisha, J., Richards, J. A., Binder, M. D., Probst, M., & Stubbs, B. (2017) · Disability and Rehabilitation · doi:10.1080/09638288.2017.1306587 · Cited by 109 · see citing works
“The acceptability of the psychometric properties of measurement tools was assessed according to previous recommendations [31].”
BackgroundMethod
In search for a public health leadership competency framework to support leadership curriculum-a consensus study
Czabanowska, K., Smith, T., Könings, K. D., Šumskas, L., Otok, R., Bjegovic-Mikanovic, V. M., & Brand, H. (2013) · European Journal of Public Health · doi:10.1093/eurpub/ckt158 · Cited by 113 · see citing works
Individualized care scale – nurse version: a Finnish validation study
Suhonen, R., Gustafsson, M., Katajisto, J., Valimaki, M. A., & Leino‐Kilpi, H. (2010) · Journal of Evaluation in Clinical Practice · doi:10.1111/j.1365-2753.2009.01168.x · Cited by 95 · see citing works
Perceived school climate and adolescent Internet addiction: The mediating role of deviant peer affiliation and the moderating role of effortful control
Li, D., Zhou, Y., Li, X., & Zhou, Z. (2016) · Computers in Human Behavior · doi:10.1016/j.chb.2016.02.015 · Cited by 103 · see citing works
Integrating generative AI into STEM education: enhancing conceptual understanding, addressing misconceptions, and assessing student acceptance
Fathi, T. E., Saad, A., Larhzil, H., Lamri, D., & Ibrahmi, E. M. A. (2025) · Disciplinary and Interdisciplinary Science Education Research · doi:10.1186/s43031-025-00125-z · Cited by 127 · see citing works
“To further establish validity, item difficulty and discrimination indices were analyzed using the CTT approach (DeVon et al., 2007).”
In-service teachers’ self-perceptions of digital competence and OER use as determined by a xMOOC training course
Soledad, M., Mena, J., & Rodríguez-Arroyo, J. A. (2017) · Computers in Human Behavior · doi:10.1016/j.chb.2017.09.010 · Cited by 107 · see citing works
“To assure that the items reflected the range of attributes of the two constructs, the researchers first established the conceptual framework and 11 drafted twenty items – ten for each construct (DeVon et al., 2007).”
Background
Sustainable Agri-food Supply Chain Practices: Few Empirical Evidences from a Developing Economy
Joshi, S., Singh, R. K., & Sharma, M. (2020) · Global Business Review · doi:10.1177/0972150920907014 · Cited by 103 · see citing works
“Content validity requires review of questionnaire from seven or more experts (DeVon et al., 2007).”
Background
Nurses quietly quit their job more often than other healthcare workers: An alarming issue for healthcare services.
Galanis, P., Katsiroumpa, A., Vraka, I., Siskou, O., Konstantakopoulou, O., Katsoulas, T., Moisoglou, I., Gallos, P., & Kaitelidou, D. (2024) · International Nursing Review · doi:10.21203/rs.3.rs-3100000/v1 · Cited by 89
“Further, we estimated corrected item-total correlations with values between 0.15 to 0.75 considered as acceptable (DeVon et al., 2007).”
MethodInfluential
Instrument development and psychometric testing in nursing education research
Mikkonen, K., Tomietto, M., & Watson, R. A. (2022) · Nurse Education Today · doi:10.1016/j.nedt.2022.105603 · Cited by 85 · see citing works
The Impact of Coping Strategies on Mental and Physical Well-Being in Patients with Rheumatoid Arthritis
Englbrecht, M., Gossec, L., DeLongis, A., Voshaar, M., Sokka, T., Kvien, T. K., & Schett, G. A. (2012) · Seminars in Arthritis and Rheumatism · doi:10.1016/j.semarthrit.2011.07.009 · Cited by 70 · see citing works
Towards a unified criteria model for usability evaluation in the context of open source software based on a fuzzy Delphi method
Dawood, K. A., Sharif, K. Y., Ghani, A., Zulzalil, H. B., Zaidan, A. A., & Zaidan, B. B. (2020) · Information and Software Technology · doi:10.1016/j.infsof.2020.106453 · Cited by 86 · see citing works
Screening for caregivers at risk: Extended validation of the short version of the Burden Scale for Family Caregivers (BSFC-s) with a valid classification system for caregivers caring for an older person at home
Pendergrass, A., Malnis, C., Graf, U., Engel, S., & Graessel, E. (2018) · BMC Health Services Research · doi:10.1186/s12913-018-3047-4 · Cited by 91 · see citing works
The cancer Support Person's Unmet Needs Survey
Campbell, H., Sanson-Fisher, R., Taylor-Brown, J., Hayward, L., Wang, X., & Turner, D. (2009) · Cancer · doi:10.1002/cncr.24386 · Cited by 83 · see citing works
Assessing the construct validity and reliability of the parental perception on antibiotics (PAPA) scales
Alumran, A. K., Hou, X., Sun, J., Yousef, A. A., & Hurst, C. P. (2014) · BMC Public Health · doi:10.1186/1471-2458-14-73 · Cited by 81 · see citing works
Are Active Australia physical activity questions valid for older adults?
Heesch, K. C., Hill, R. L., van Uffelen, J. G. Z., & Brown, W. J. (2011) · Journal of Science and Medicine in Sport · doi:10.1016/j.jsams.2010.11.004 · Cited by 80 · see citing works
“It should be noted that summary PA measures in this study did not account for the intensity of the PA. Typically with AAS data,28 PA scores are computed by multiplying the minutes in each PA by an assigned MET value to account for differences in intensity among the three PA types (walking*3.0 METs, moderate-intensity…”
BackgroundInfluential
Validity and reliability of instruments designed to measure factors influencing the overuse of antibiotics
Alumran, A. K., Hou, X., & Hurst, C. P. (2012) · Journal of Infection and Public Health · doi:10.1016/j.jiph.2012.03.003 · Cited by 69 · see citing works
“Reliability can be assessed by confirming an instrument’s ability to measure a consistent attribute [1].”
BackgroundInfluential
Assessing the carbon footprint of digital health interventions: a scoping review
Lokmic‐Tomkins, Z., Davies, S., Block, L. J., Cochrane, L., Dorin, A., von Gerich, H., Lozada‐Perezmitre, E., Reid, L., & Peltonen, L. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocac196 · Cited by 105 · see citing works
Clinical learning environment and supervision of international nursing students: A cross-sectional study
Mikkonen, K., Elo, S., Miettunen, J., Saarikoski, M., & Kääriäinen, M. (2017) · Nurse Education Today · doi:10.1016/j.nedt.2017.02.017 · Cited by 80 · see citing works
Development and Validation of a Student Self-efficacy Scale
Schmitz, G. S. (2013) · Journal of Nursing & Care · doi:10.4172/2167-1168.1000126 · Cited by 63 · see citing works
“Validity is confirmed when scores on a scale are strongly correlated to a related criterion or scale at the same point in time [26].”
Herbal medicines: a cross-sectional study to evaluate the prevalence and predictors of use among Jordanian adults
El‐Dahiyat, F., Rashrash, M., Abuhamdah, S. M. A., Farha, R. K. A., & Babar, Z. (2020) · Journal of Pharmaceutical Policy and Practice · doi:10.1186/s40545-019-0200-3 · Cited by 108 · see citing works
Psychometric Analysis of the Heart Failure Somatic Perception Scale as a Measure of Patient Symptom Perception
Jurgens, C. Y., Lee, C. S., & Riegel, B. J. (2015) · The Journal of Cardiovascular Nursing · doi:10.1097/jcn.0000000000000320 · Cited by 90 · see citing works
“To support discriminant validity, correlation between 2 different constructs should be low.(26,27) The Self-care of HF Index (SCHFI) was used to quantify self-care.”
Basic competence in intensive and critical care nursing: development and psychometric testing of a competence scale.
Lakanmaa, R.-L., Suominen, T., Perttilä, J., Ritmala-Castren, M., Vahlberg, T., & Leino-Kilpi, H. (2014) · Journal of Clinical Nursing · doi:10.1111/jocn.12057 · Cited by 77
Validation of the Turkish version of the foot and ankle outcome score
Karatepe, A. G., Günaydın, R., Kaya, T., Karlıbaş, U., & Özbek, G. (2009) · Rheumatology International · doi:10.1007/s00296-009-0929-0 · Cited by 79 · see citing works
“7 is generally regarded as satisfactory [15].”
Public knowledge of chronic kidney disease evaluated using a validated questionnaire: a cross-sectional study
Gheewala, P. A., Peterson, G. M., Zaidi, S. T. R., Jose, M., & Castelino, R. L. (2018) · BMC Public Health · doi:10.1186/s12889-018-5301-4 · Cited by 102 · see citing works
Pancreatitis quality of life instrument: A psychometric evaluation
Wassef, W., DeWitt, J., McGreevy, K., Wilcox, M., Whitcomb, D., Yadav, D., Amann, S., Mishra, G., Alkaade, S., Romagnuolo, J., et al. (2016) · American Journal of Gastroenterology · doi:10.1038/ajg.2016.225 · Cited by 73 · see citing works
Healthcare professionals’ digital health competence and its core factors; development and psychometric testing of two instruments
Jarva, E., Oikarinen, A., Andersson, J., Tomietto, M., Kääriäinen, M., & Mikkonen, K. (2023) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2023.104995 · Cited by 78 · see citing works
Psychometric properties of the health-promoting lifestyle profile II: cross-cultural validation of the Malay language version
Kuan, G., Kueh, Y. C., Abdullah, N., & Tai, E. L. M. (2019) · BMC Public Health · doi:10.1186/s12889-019-7109-2 · Cited by 77 · see citing works
Social networking time use scale (SONTUS): A new instrument for measuring the time spent on the social networking sites
Olufadi, Y. (2015) · Telematics and Informatics · doi:10.1016/j.tele.2015.11.002 · Cited by 86 · see citing works
Physicians’ attitudes and knowledge concerning antibiotic prescription and resistance: questionnaire development and reliability
Rodrigues, A. T., Ferreira, M., Roque, F., Falcão, A. C., Ramalheira, E., Figueiras, A., & Herdeiro, M. T. (2015) · BMC Infectious Diseases · doi:10.1186/s12879-015-1332-y · Cited by 82 · see citing works
“Face validity, as a form of validity that refers to a subjective assessment [42], and content validity, as a measure of the comprehensiveness and representativeness of a scale’s content [19], were assessed and guaranteed by the panels of experts, the literature review [12] and a qualitative study conducted by a collab…”
Method
Validity and reliability testing of a short questionnaire developed to assess consumers’ use, understanding and perception of food labels
Mackison, D., Wrieden, W. L., & Anderson, A. S. (2009) · European Journal of Clinical Nutrition · doi:10.1038/ejcn.2009.126 · Cited by 77 · see citing works
Psychometric Evaluation of the Summary of Diabetes Self-Care Activities–Arabic (SDSCA-Arabic)
Aljohani, K. A., Kendall, G. E., & Snider, P. D. (2014) · Journal of Transcultural Nursing · doi:10.1177/1043659614526255 · Cited by 66 · see citing works
“The minimum acceptable alpha score for the internal consistency was .70 (DeVon et al., 2007).”
Background
Cross-cultural adaptation and validation of the South African Pain Catastrophizing Scale (SA-PCS) among patients with fibromyalgia
Morris, L. D., Grimmer, K. A., Louw, Q. A., & Sullivan, M. J. L. (2012) · Health and Quality of Life Outcomes · doi:10.1186/1477-7525-10-137 · Cited by 69 · see citing works
The use of the EQ-5D-Y health related quality of life outcome measure in children in the Western Cape, South Africa: psychometric properties, feasibility and usefulness - a longitudinal, analytical study
Scott, D., Ferguson, G. D., & Jelsma, J. M. (2017) · Health and Quality of Life Outcomes · doi:10.1186/s12955-017-0590-3 · Cited by 86 · see citing works
Workload scoring systems in the Intensive Care and their ability to quantify the need for nursing time: A systematic literature review
Hoogendoorn, M. E., Margadant, C. C., Brinkman, S., Haringman, J. J., Spijkstra, J. J., & de Keizer, N. F. (2019) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2019.103408 · Cited by 73 · see citing works
Psychometric properties of three single-item pain scales in patients with rheumatoid arthritis seen during routine clinical care: a comparative perspective on construct validity, reproducibility and internal responsiveness
Sendlbeck, M., Araujo, E. G., Schett, G. A., & Englbrecht, M. (2015) · RMD Open · doi:10.1136/rmdopen-2015-000140 · Cited by 65 · see citing works
“70 were considered reliable.(26) In patients undergoing a change in antirheumatic therapy, for example, owing to side effects or lack of efficacy, we investigated the internal responsiveness of the pain scales to changes of antirheumatic treatment by Figure 1 The visual analogue scale, the numerical rating scale and t…”
Method
Health system context and implementation of evidence-based practices—development and validation of the Context Assessment for Community Health (COACH) tool for low- and middle-income settings
Bergström, A., Skeen, S., Đức, D. M., Blandón, E. Z., Estabrooks, C. A., Gustavsson, P. J., Hoa, D. T. P., Källestål, C., Målqvist, M., Nga, N. T., et al. (2015) · Implementation Science · doi:10.1186/s13012-015-0305-2 · Cited by 74 · see citing works
“Following retention of items, we examined corrected total item correlation and average inter-item correlation [53, 54].”
Method
Assessing safety climate in acute hospital settings: a systematic review of the adequacy of the psychometric properties of survey measurement tools
Salem, G. A., Bowie, P., & Morrison, J. M. (2018) · BMC Health Services Research · doi:10.1186/s12913-018-3167-x · Cited by 76 · see citing works
“Conversely, Discriminant Validity represents the extent to which measures of different constructs show correlation with one other [78].”
BackgroundMethod
Business and innovation resources: Determinants for the survival of new technology-based firms
Löfsten, H. (2016) · Management Decision · doi:10.1108/md-04-2015-0139 · Cited by 66 · see citing works
Investigating the role of an enterprise architecture project in the business-IT alignment in Iran
Alaeddini, M., & Salekfard, S. (2011) · Information Systems Frontiers · doi:10.1007/s10796-011-9332-y · Cited by 66
“The reliability of a study design refers to the ability of an instrument to consistently measure an attribute (DeVon et al .”
Background
Validation of the UNESP-Botucatu pig composite acute pain scale (UPAPS)
Luna, S., de Araújo, A. L., da Nóbrega, P. I., Brondani, J., de Oliveira, F. A., Azerêdo, L. M. D. S., Telles, F. G., & Trindade, P. (2020) · PLoS ONE · doi:10.1371/journal.pone.0233552 · Cited by 65 · see citing works
“The dimensions and domains of the tool were determined by principal component analysis, which is normally used to relate the tool’s variables in a grouped manner [62].”
Method
Determining the validity and reliability of the Chinese version of the Kidney Disease Quality of Life Questionnaire (KDQOL-36™)
Tao, X., Chow, S. K. Y., & Wong, F. K. Y. (2014) · BMC Nephrology · doi:10.1186/1471-2369-15-115 · Cited by 67 · see citing works
The Communicative and Critical Health Literacy Scale – Swedish Version
Wångdahl, J. M., & Mårtensson, L. I. (2013) · Scandinavian Journal of Public Health · doi:10.1177/1403494813500592 · Cited by 66 · see citing works
“To make sufficient evaluations of the use of interventions, reliable and valid instruments are needed [3].”
Background
Instruments measuring interprofessional collaboration in healthcare – a scoping review
Peltonen, J., Leino‐Kilpi, H., Heikkilä, H. C., Rautava, P. T., Tuomela, K., Siekkinen, M., Sulosaari, V., & Stolt, M. (2019) · Journal of Interprofessional Care · doi:10.1080/13561820.2019.1637336 · Cited by 79 · see citing works
“As instrument development and testing is a lengthy process (DeVon et al., 2007), their further testing might be still under progress.”
BackgroundResult
Science Parks, talent attraction and stakeholder involvement: an international study
Cadorin, E., Klofsten, M., & Löfsten, H. (2019) · Journal of Technology Transfer · doi:10.1007/s10961-019-09753-w · Cited by 63
“700 is acceptable for a new instrument (DeVillis 1991; DeVon et al. 2007).”
Background
Measuring Pain and Efficacy of Pain Treatment in Inflammatory Arthritis: A Systematic Literature Review
Englbrecht, M., Tarner, I. H., van der Heijde, D. M. F. M., Manger, B., Bombardier, C., & Müller‐Ladner, U. (2012) · Journal of Rheumatology Supplement · doi:10.3899/jrheum.120335 · Cited by 64 · see citing works
Development and psychometric testing of a Clinical Reasoning Evaluation Simulation Tool (CREST) for assessing nursing students' abilities to recognize and respond to clinical deterioration
Liaw, S. Y., Rashasegaran, A., Wong, L. F., Deneen, C. C., Cooper, S., Levett-Jones, T., Goh, H. S., & Ignacio, J. (2018) · Nurse Education Today · doi:10.1016/j.nedt.2017.12.009 · Cited by 65 · see citing works
Psychometric Properties of the Persian Language Version of Yang Internet Addiction Questionnaire: An Explanatory Factor Analysis
Mohammadsalehi, N., Mohammadbeigi, A., Jadidi, R., Anbari, Z., Ghaderi, E., & Akbari, M. (2015) · international journal high risk behaviors & addiction · doi:10.5812/ijhrba.21560 · Cited by 58 · see citing works
Validation of the Social Inclusion Scale with Students
Wilson, C., & Secker, J. (2015) · Social Inclusion · doi:10.17645/si.v3i4.121 · Cited by 70 · see citing works
“The mean scores of the two groups should differ significantly in the expected direction if the instrument is valid (e.g. DeVon et al., 2007).”
Background
PENGUJIAN VALIDASI ISI (CONTENT VALIDITY) ANGKET PERSEPSI MAHASISWA TERHADAP PEMBELAJARAN DARING MATAKULIAH MATEMATIKA KOMPUTASI
Puspitasari, W., & Febrinita, F. (2021) · Factor M · doi:10.30762/factor_m.v4i1.3254 · Cited by 59 · see citing works
Quantitative and mixed analyses to identify factors that affect cervical cancer screening uptake among lesbian and bisexual women and transgender men
Johnson, M. J., Mueller, M., Eliason, M. J., Stuart, G. W., & Nemeth, L. S. (2016) · Journal of Clinical Nursing · doi:10.1111/jocn.13414 · Cited by 60 · see citing works
Cloud computing services adoption among higher education faculties: development of a standardized questionnaire
Asadi, Z., Abdekhoda, M., & Nadrian, H. (2019) · Education and Information Technologies · doi:10.1007/s10639-019-09932-0 · Cited by 65 · see citing works
Assessing patient reported outcome measures: A practical guide for gastroenterologists
Alrubaiy, L., Hutchings, H. A., & Williams, J. G. (2014) · United European Gastroenterology Journal · doi:10.1177/2050640614558345 · Cited by 65 · see citing works

Authors: DeVon, H. A., Block, M. E., Moyle‐Wright, P., Ernst, D. M., Hayden, S. J., Lazzara, D. J., Savoy, S. M., & Kostas‐Polston, E.
Affiliations: Purdue University Northwest (US)
Corresponding author: DeVon, Holli A.
Block, Michelle E.
Moyle‐Wright, Patricia
Ernst, Diane M.
Hayden, Susan J.
Lazzara, Deborah J.
Savoy, Suzanne M.
ORCID: DeVon, Holli A.
Venue: Journal of Nursing Scholarship
Publisher: Wiley
Published: 2007-05-23
Volume: 39
Issue: 2
Pages: 155-164
ISSN: 0363-2792 (ISSN-L)
1527-6546
1547-5069
DOI: 10.1111/j.1547-5069.2007.00161.x
OpenAlex: W2162602057
Cited by: 1,889
References: 0
Citations by year: 2026: 87 · 2025: 128 · 2024: 186 · 2023: 176 · 2022: 148 · 2021: 163 · 2020: 145 · 2019: 144 · 2018: 131 · 2017: 113 · 2016: 96 · 2015: 105 · 2014: 83 · 2013: 51 · 2012: 40
FWCI: 15.03
Type: journal article
Language: en
Open access: closed
Licence: onlinelibrary.wiley.com/termsAndConditions#vor (vor)
Locations: Journal of Nursing Scholarship (publishedVersion)
PubMed (publishedVersion)
ePublications at Regis University (Regis University) (submittedVersion)
Topics: Social Sciences › Education › Health Education and Validation
Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
Keywords: psychometric properties, reliability statistics, validity, content validity, criterion validity, construct validity, internal consistency, test-retest reliability, measurement error, nursing equipment, nursing research
MeSH terms: Data Interpretation, Statistical, Factor Analysis, Statistical, Humans, Mathematics, Periodicals as Topic, Planning Techniques, Predictive Value of Tests, Psychometrics, Research Design, Time Factors, Reproducibility of Results, Nursing Research, Bias, Discriminant Analysis, Guidelines as Topic, Peer Review, Research, Statistics, Nonparametric, Editorial Policies

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

A Psychometric Toolbox for Testing Validity and Reliability

Abstract

Integrity signals

Claims the paper makes

Claims other papers make about this one

Cites

Cited by

Record

About these checks