Establishing survey validity: A practical guide
Open accessAbstract
What follows is a practical guide for establishing the validity of a survey for research purposes. The motivation for providing this guide is our observation that researchers, not necessarily being survey researchers per se, but wanting to use a survey method, lack a concise resource on validity. There is far more to know about surveys and survey construction than what this guide provides; and this guide should only be used as a starting point. However, for the needs of many researchers, this guide provides sufficient, basic information on survey validity. The guide, furthermore, includes references to important handbooks for researchers needing further information.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The survey design was supported by pilot testing and content validity expert review prior to data collection.
“The survey design was supported by pilot testing and content validity expert review (Cobern et al., 2020) prior to data collection.”
From Mentorship in the Age of Generative AI: ChatGPT to Support Self-Regulated Learning of Pre-Service Teachers Before and During Placements (2025) · cites this paper as Cobern et al., 2020
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The Use of Cronbach’s Alpha When Developing and Reporting Research Instruments in Science Education
Taber, K. S. (2018) · Research in Science Education · doi:10.1007/s11165-016-9602-2 · Cited by 10,679 · see citing works
Interviewing as qualitative research a guide for researchers in education and the social sciences
Seidman, I. (2006) · Cited by 9,900 · see citing works
The standards for educational and psychological testing.
Eignor, D. R. (2013) · American Psychological Association eBooks · doi:10.1037/14047-013 · Cited by 6,975 · see citing works
Boateng, G. O., Neilands, T. B., Frongillo, E. A., Melgar-Quiñonez, H. R., & Young, S. L. (2018) · Frontiers in Public Health · doi:10.3389/fpubh.2018.00149 · Cited by 5,925 · cited 1× · see citing works
“‡ Boateng et al. (2018) offers a similar practical guide but from a different perspective with somewhat different coverage.”
BackgroundInternet, Phone, Mail, and Mixed‐Mode Surveys
Dillman, D. A., Smyth, J. D., & Christian, L. M. (2014) · Wiley eBooks · doi:10.1002/9781394260645 · Cited by 4,484 · see citing works
Rasch Analysis for Instrument Development: Why, When, and How?
Boone, W. J. (2016) · CBE—Life Sciences Education · doi:10.1187/cbe.16-04-0148 · Cited by 701 · see citing works
Measuring Cognitive Engagement With Self-Report Scales: Reflections From Over 20 Years of Research
Greene, B. A. (2015) · Educational Psychologist · doi:10.1080/00461520.2014.989230 · Cited by 452 · see citing works
The SAGE Handbook of Survey Methodology
Wolf, C., Joye, D., Smith, T., et al. (2016) · SAGE Publications Ltd eBooks · doi:10.4135/9781473957893 · Cited by 366 · see citing works
The Practice of Survey Research: Theory and Applications
Ruel, E. E., Wagner, W. E., & Gillespie, B. J. (2016) · Sage eBooks · doi:10.4135/9781483391700 · Cited by 266 · see citing works
The Palgrave Handbook of Survey Research
(2017) · Springer eBooks · doi:10.1007/978-3-319-54395-6 · Cited by 180 · see citing works
Faculty Perspectives of Undergraduate Chemistry Laboratory: Goals and Obstacles to Success
Bruck, L. B., Towns, M. H., & Bretz, S. L. (2010) · Journal of Chemical Education · doi:10.1021/ed900002d · Cited by 149 · see citing works
Sample Size for Cognitive Interview Pretesting
Blair, J., & Conrad, F. G. (2011) · Public Opinion Quarterly · doi:10.1093/poq/nfr035 · Cited by 143 · see citing works
MEASURING SCIENCE INTEREST: RASCH VALIDATION OF THE SCIENCE INTEREST SURVEY
Lamb, R. L., Annetta, L. A., Meldrum, J., & Vallett, D. (2011) · International Journal of Science and Mathematics Education · doi:10.1007/s10763-011-9314-z · Cited by 131 · see citing works
Bruck, A. D., & Towns, M. H. (2013) · Journal of Chemical Education · doi:10.1021/ed300371n · Cited by 102 · see citing works
Zoller, U. (1999) · Journal of Research in Science Teaching · doi:10.1002/(sici)1098-2736(199905)36:5 · Cited by 66 · see citing works
Self-citationPedagogy of Science Teaching Tests: Formative assessments of science teaching orientations
Cobern, W. W., Schuster, D., Adams, B. A., Skjold, B. A., Muğaloğlu, E. Z., Bentz, A., & Sparks, K. (2014) · International Journal of Science Education · doi:10.1080/09500693.2014.918672 · Cited by 65 · see citing works
Self-citationImproving Question Design to Maximize Reliability and Validity
Krosnick, J. A. (2017) · Springer eBooks · doi:10.1007/978-3-319-54395-6_13 · Cited by 63 · see citing works
When interviewing: how many is enough?
Cobern, W. W., & Adams, B. A. (2020) · International Journal of Assessment Tools in Education · doi:10.21449/ijate.693217 · Cited by 59 · see citing works
Self-citationWillis, G. B. (2016) · The SAGE Handbook of Survey Methodology · doi:10.4135/9781473957893.n24 · Cited by 59 · see citing works
Cognitive Interviewing in Survey Design: State of the Science and Future Directions
Willis, G. (2017) · Springer eBooks · doi:10.1007/978-3-319-54395-6_14 · Cited by 53 · see citing works
Validation of a Measure of STEM Interest for Adolescents
Staus, N. L., Lesseig, K., Lamb, R. L., Falk, J. H., & Dierking, L. D. (2019) · International Journal of Science and Mathematics Education · doi:10.1007/s10763-019-09970-7 · Cited by 38 · see citing works
Development and application of a scale to measure students’ STEM continuing motivation
Luo, T., Wang, J., Liu, X., & Zhou, J. (2019) · International Journal of Science Education · doi:10.1080/09500693.2019.1647472 · Cited by 34 · see citing works
Cobern, W. W. (1995) · Cited by 25 · see citing works
Self-citationDesigning Questions and Questionnaires
Smyth, J. D. (2016) · SAGE Publications Ltd eBooks · doi:10.4135/9781473957893.n16 · Cited by 21 · see citing works
Clarifying Categorical Concepts in a Web Survey
Redline, C. D. (2013) · Public Opinion Quarterly · doi:10.1093/poq/nfs067 · Cited by 14 · see citing works
Haryani, E., Cobern, W. W., & Pleasants, B. A.-S. (2019) · Journal of Research in Science Mathematics and Technology Education · doi:10.31756/jrsmte.224 · Cited by 6 · see citing works
Self-citationGender and Conceptualizations of Nature
Cobern, W. W. (2000) · Springer eBooks · doi:10.1007/978-94-011-4171-0_7 · Cited by 1 · see citing works
Self-citationCobern, W. W., Adams, B. A., Pleasants, B. A.-S., Bentley, A. P. K., & Kagumba, R. (2019) · ScholarWorks - WMU (Western Michigan University) · Cited by 1 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 3
- Cobern, W. W., Gibson, A. T., & Underwood, S. A. (1999). Conceptualizations of nature: An interpretive study of 16 ninth graders' everyday thinking. Journal of Research in Science Teaching, 36(5), 541-564.
- Multon, K. D. (2010). Test-retest reliability. In N. J. Salkind (Ed.), Encyclopedia of research design (pp. 1495-1498). Thousand Oaks, California: SAGE Publications, Inc.
- Nyutu, E. N., Cobern, W. W., & Pleasants, B. A.-S. (2020). Development of an instrument to assess students' perceptions of their undergraduate laboratory environment. The Journal for Research and Practice in College Teaching, 5(1), 1-18. Retrieved from https://journals.uc.edu/index.php/jrpct/article/view/1492
Cited by
Sorted by citations
STEM Learning Environment: Perceptions and Implementation Skills in Prospective Science Teachers
Rusydiyah, E. F., Indarwati, D., Jazil, S., Susilawati, S., & Gusniwati, G. (2021) · Jurnal Pendidikan IPA Indonesia · doi:10.15294/jpii.v10i1.28303 · Cited by 50 · see citing works
Nguyen (Ruby), N. N., & Barbieri, W. (2025) · Education Sciences · doi:10.3390/educsci15060642 · Cited by 15 · see citing works
Kennedy, H. M., Cole, A., Berbert, L., Schenkel, S. R., & DeGrazia, M. (2023) · Child Care Health and Development · doi:10.1111/cch.13206 · Cited by 12 · see citing works
Marques, M., Ferreira-Santos, J., Rodrigues, R., & Pombo, L. (2025) · Computers · doi:10.3390/computers14070267 · Cited by 14 · see citing works
Chi, L., Boucaut, R., Li, L. S. K., Fryer, C. E., & Kumar, S. (2024) · Physiotherapy Research International · doi:10.1002/pri.2085 · Cited by 11 · see citing works
Coyte, E. (2023) · FEBS Open Bio · doi:10.1002/2211-5463.13687 · Cited by 12 · see citing works
Colicol, F. L., & Sali-Latif, F. K. (2023) · International Journal of Learning Teaching and Educational Research · doi:10.26803/ijlter.22.6.19 · Cited by 9 · see citing works
The Art Nouveau Path: Promoting Sustainability Competences Through a Mobile Augmented Reality Game
Ferreira-Santos, J., & Pombo, L. (2025) · Multimodal Technologies and Interaction · doi:10.3390/mti9080077 · Cited by 8 · see citing works
Patients’ and healthcare professionals’ experience with at-home parenteral therapy
Puzovic, M., Morrissey, H., & Ball, P. (2023) · Journal Of Advanced Pharmacy Education And Research · doi:10.51847/shh6wzi9tf · Cited by 8 · see citing works
Ziegler-Rodríguez, K., Josa, I., Castro, L., Escalante, H., Vera-Mercado, E., & Garfí, M. (2025) · Sustainable Production and Consumption · doi:10.1016/j.spc.2025.01.004 · Cited by 7 · see citing works
Mahdavi, A., Miño-Rodríguez, I., Berger, C., Martínez-Muñoz, I., & Wagner, A. (2024) · Building Research & Information · doi:10.1080/09613218.2024.2350563 · Cited by 7 · see citing works
Lepore, T., Hlusko, L. J., Armstrong, L. B., Frank, T., Tseng, Z. J., Schmitt, C. A., Rizk, O. T., & Baranger, A. M. (2025) · CBE—Life Sciences Education · doi:10.1187/cbe.24-12-0295 · Cited by 5 · see citing works
Energy and Carbon Savings in European Households Resulting from Behavioral Changes
Widera, B. (2024) · Energies · doi:10.3390/en17163888 · Cited by 5 · see citing works
Ekren, E., Hall, R. W., Pierdolla, E. L., Barnes, V., Jarzombek-Torralva, A., Morrish, D. G., & Martinez-Prather, K. E. (2025) · Safety Science · doi:10.1016/j.ssci.2025.106781 · Cited by 4 · see citing works
Developing teacher leader efficacy to address attrition
Lin, J., Galosy, J. A., Ross, C., Watts, J., & Wilson, G. (2025) · School Science and Mathematics · doi:10.1111/ssm.18338 · Cited by 5 · see citing works
Ferreira-Santos, J., & Pombo, L. (2025) · Sustainability · doi:10.3390/su17188150 · Cited by 4 · see citing works
Development of a Mexican Version of the Cycle-Tourist Motivation Instrument (CtMI)
Mladosich, R. A. A., Trujillo, R. A. M.-M., Hidalgo‐Contreras, J. V., & Becerra-Roman, I. (2022) · Sustainability · doi:10.3390/su142113866 · Cited by 4 · see citing works
Assessment tools and strategies used by Jamaican secondary school teachers
Williams-McBean, C. T. (2022) · International Journal of Assessment Tools in Education · doi:10.21449/ijate.980870 · Cited by 4 · see citing works
Assessment of a Survey Instrument for Measuring Affective Pathways
Treadway, E., Tubbs, K., Caserto, M., Lee, M., & Swenson, J. E. S. (2024) · ASEE Annual Conference & Exposition · doi:10.18260/1-2--42328 · Cited by 3 · see citing works
Identifying Barriers to Precepting Health Professions Learners During the COVID-19 Pandemic
Lund, M. B., Green, S., Leppke, A., Vohra-Khullar, P. D., Alfonso, S. A., & Moore, M. A. (2024) · PRiMER · doi:10.22454/primer.2024.453046 · Cited by 3 · see citing works
Lin, C. A., Zailani, S. H. M., & Mahyudin, N. (2025) · Sustainable Futures · doi:10.1016/j.sftr.2025.100829 · Cited by 3 · see citing works
Chale, H. A., Ding, X., Liu, R., Chol, B. D., & Huang, H. (2026) · Land Use Policy · doi:10.1016/j.landusepol.2026.107946 · Cited by 2 · see citing works
Shameem, M., Anwer, S., Mahmood, S., Algamdi, S. A., Niazi, M., & Mohammed, M. (2026) · Applied Soft Computing · doi:10.1016/j.asoc.2026.116127 · Cited by 2 · see citing works
Ohio Health Care Professionals’ Survey: Work and Home Stressors During the COVID-19 Pandemic
McCloskey, R. J., Goldberg, L., Santucci, R., Koralewski, J., Kocinski, M., & Hammond, G. C. (2023) · Ohio Journal of Public Health · doi:10.18061/ojph.v5i2.9162 · Cited by 2 · see citing works
The Experience of Home Parenteral Therapy: A Thematic Analysis of Patient Interviews
Puzovic, M., Morrissey, H., & Ball, P. (2023) · Pharmacy · doi:10.3390/pharmacy11050133 · Cited by 2 · see citing works
Kumi, E., Osei, H. V., Korantwi-Barimah, J. S., & Kumi-Richardson, E. A. (2024) · International Journal of Innovation and Technology Management · doi:10.1142/s0219877024500524 · Cited by 2 · see citing works
Kumi, E. (2024) · African Journal Of Applied Research · doi:10.26437/ajar.v10i2.795 · Cited by 2 · see citing works
Law, L., Ingram, L. A., Boucaut, R., & Kumar, S. (2025) · Physiotherapy Research International · doi:10.1002/pri.70067 · Cited by 2 · see citing works
Febrey, S., Frost, J., Hall, A., Morley, N., Whitney, J. C., Johnston, V., Wilson, P., Kilgore, C., & Goodwin, V. A. (2024) · International Journal for Advancing Practice · doi:10.12968/ijap.2023.0048 · Cited by 2 · see citing works
Building a Structured, Evidence-Based Remediation Program for Today’s Learners
Kendall, A. V., Lund, M. B., Moore, M. A., & Alfonso, S. A. (2025) · The Journal of Physician Assistant Education · doi:10.1097/jpa.0000000000000685 · Cited by 1 · see citing works
Bartels, H. C., Fox, K. A., Aryananda, R. A., Terlizzi, K., Cooney, N., Nieto‐Calvache, A. J., Lalor, J., & Brennan, D. J. (2025) · International Journal of Gynecology & Obstetrics · doi:10.1002/ijgo.70345 · Cited by 1 · see citing works
Implementation of transcranial magnetic stimulation in speech-language pathology practice: a survey
Williams, E. E. R., Hordacre, B. G., Goldsworthy, M. R., Brocksopp, C., & Attrill, S. (2025) · Disability and Rehabilitation · doi:10.1080/09638288.2025.2543054 · Cited by 1 · see citing works
Noor, N. M., & Sutrisno, N. D. (2026) · Issues and Perspectives in Business and Social Sciences · doi:10.33093/ipbss.2026.6.2.19 · Cited by 1 · see citing works
Lufler, R. S., Saini, H., Cardamone, C. N., & Marchant, J. K. (2024) · The Journal of Physician Assistant Education · doi:10.1097/jpa.0000000000000616 · Cited by 1 · see citing works
Banda, L. O. L., Banda, J. T., & Banda, C. V. (2025) · Humanities and Social Sciences Communications · doi:10.1057/s41599-025-05651-9 · Cited by 1 · see citing works
Ohio Journal of Public Health Vol. 5, Issue 2 (July 2023): Full Issue
Ohio Public Health Association OPHA (2023) · Ohio Journal of Public Health · doi:10.18061/ojph.v5i2.9598 · Cited by 1 · see citing works
Sattar, A., Khurram, A. F. A., & Asif, M. (2023) · Pakistan Journal of Humanities and Social Sciences · doi:10.52131/pjhss.2023.1102.0514
Campus climate assessment and action: disaggregating the social work experience in Canada
Curry‐Stevens, A., Petovello, A., & Hayford, E. (2024) · Social Work Education · doi:10.1080/02615479.2024.2389331
Assessment of Journalism Principles in Media: An Evaluative Instrument
Fisher, J. A., & George-Palilonis, J. A. (2024) · Journalism Practice · doi:10.1080/17512786.2024.2394551
Moran, V., Moore, S. G., & Ho, P. (2025) · Journal of Human Nutrition and Dietetics · doi:10.1111/jhn.70029
Medhekar, R., Clark, L., Gautam, S., & Cortoos, A. (2025) · Drug Delivery · doi:10.1080/10717544.2025.2537813
Teachers’ perceptions of literacy instruction for their learners with autism spectrum disorder
Human, H. J. W., Graham, M. A., & Bornman, J. (2025) · South African Journal of Childhood Education · doi:10.4102/sajce.v14i1.1722
A survey on experiences of ward rounds among inpatients in Singapore acute care hospitals
Seah, Q. H. V., Rajasegeran, D. D., Fan, P. E. M., Lim, P. Y. C., Ayre, T. C., & Ang, S. Y. (2025) · Proceedings of Singapore Healthcare · doi:10.1177/20101058251380381
Tucker, V. M., & Charette, C. (2025) · Open Praxis · doi:10.55982/openpraxis.17.4.930
Stankova, M., Ivanov, I., Mihova, P., & Proedrou, A. (2026) · Journal of Intellectual Disability - Diagnosis and Treatment · doi:10.6000/2292-2598.2026.14.01.1
Montilla‐Herrador, J., Gacto‐Sánchez, M., Lozano-Meca, J. A., Martínez-González, M., Marín‐Sánchez, P., & Medina‐Mirapeix, F. (2026) · Journal of Clinical Medicine · doi:10.3390/jcm15072765
Teodorescu, D., Andrei, T., & Aivaz, K.-A. (2026) · IGI Global eBooks · doi:10.4018/979-8-3373-3932-0.ch009
Roach, Q. E., & Cobern, W. W. (2026) · Journal of Geoscience Education · doi:10.1080/10899995.2026.2660360
Self-citationA Systematic Review of the Characterization of Behavior in Canine Behavioral Questionnaires
Johnson, A., Schenke, K. C., Foster, J., & Stephens-Lewis, D. (2026) · Journal of Applied Animal Welfare Science · doi:10.1080/10888705.2026.2693645
Exploring Usage Patterns of News and Information Sources among Ohio Residents
Thapa, M. P., Lawson, C. R., Rumble, J. N., Specht, A. R., & Buck, E. B. (2026) · Journal of Agricultural Education · doi:10.5032/jae.v67i3.3349
Cardamone, K., Byham-Gray, L. D., Sackey, J. D., & Brody, R. A. (2026) · Journal of the Academy of Nutrition and Dietetics · doi:10.1016/j.jand.2026.156432
Methods for Developing a Survey to Measure Engineering Judgement Across Undergraduate Coursework
Radel, H., & Swenson, J. E. S. (2026) · ASEE Annual Conference & Exposition · doi:10.18260/1-2--60157
Simunović, S., Tomasevic, I. B., Stojković, B., Rašeta, M., Bajčić, A., Simunović, S., & Djekic, I. V. (2026) · Foods · doi:10.3390/foods15172984
Mahundu, F., Kwesigabo, E., & Lugwisha, N. (2026) · Journal of Business and Social Review in Emerging Economies · doi:10.26710/jbsee.v12i3.3746
Wong, Y. Y. Y., Thompson, G. A., & Bolger, L. E. (2026) · Journal of Music Therapy · doi:10.1093/jmt/thag019
- Authors
- COBERN, W., & ADAMS, B.
- Affiliations
- Western Michigan University (US)
- Venue
- International Journal of Assessment Tools in Education
- Published
- 2020-09-15
- Volume
- 7
- Issue
- 3
- Pages
- 404-419
- ISSN
- 2148-7456 (ISSN-L)
- OpenAlex
- W3080779873
- Cited by
- 59
- References
- 27
- Citations by year
- 2026: 15 · 2025: 21 · 2024: 11 · 2023: 8 · 2022: 3 · 2021: 1
- FWCI
- 4.00
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- Locations
- International Journal of Assessment Tools in Education (publishedVersion, cc-by)
- DergiPark (Istanbul University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Topics
- Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Education › Health Education and Validation
- Keywords
- survey validity, survey design, research survey, measurement validity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
